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Much research work, particularly in developing countries, at present goes 
unreported. The reasons for this include the intense competition to publish in the 
scientific press, and difficulties in matching the research resources of developed 
countries. The DAP research series was established to provide a forum for the 
rapid distribution of data and findings relevant to critical areas of drug policy and 
use. The Action Programme has a firm commitment to national operational 
research as part of its direct country support. It is also strongly committed to 
making the findings of such studies widely known and accessible. While every 
effort is: made by the Programme to support studies of the highest possible quality, 
research skills and resources will vary from country to country. Documents in the 
DAP research series reflect this variation, and range from reports of very small 
scale studies, undertaken with minimal resources, to major global research 
involving substantive financial, scientific and editorial input. 
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Executive summary 


Executive summary 


All countries in the world have national pharmaceutical policies, either explicit 
in the form of policy documents or implicit in government actions. Very few 
studies have been carried out which try to compare national drug policies and to 
understand which policies have been successful - and why - in achieving the 
goals of availability, accessibility, good quality and rational use of drugs. This 
document describes the preliminary findings of such a study as discussed in a 
workshop held in June 1996 in WHO, Geneva. 


The comparative analysis of NDPs: Objectives and approaches 


The objectives of the research were: 


e to identify strengths, weaknesses, and political dimensions of 
pharmaceutical policy formulation and implementation within each 
country; 

* to propose explanations for cross national variations in performance; and 

° to propose effective strategies, both national and international, that can 


improve pharmaceutical policy implementation. 


The research initiated by the WHO Action Programme on Essential Drugs in 
collaboration with the Division of International Health Care Research (IHCAR), 
Stockholm, and the Harvard School of Public Health (Boston) was conducted by 
national research teams in 12 countries (Bulgaria, Chad, Colombia, Guinea, 
India, Mali, Philippines, Sri Lanka, Thailand, Viet Nam, Zambia, and 
Zimbabwe). 


To achieve its objectives, the research provided a cross national, comparative 
framework and combined formative evaluation strategies (focusing on the 
process of policy formulation and implementation) and summative evaluation 
strategies (focusing on policy outcomes). The two main research tools used were 
standardized NDP indicators - for assessment of NDP performance; and political 
mapping - for the analysis of NDP formulation and implementation processes. 


The workshop and its main outcomes 


The workshop described in the document was the third step in the 
implementation of this collaborative project. Its aim was to provide 
methodological support to the research teams, to review the results, to assess the 
two methods and to prepare a plan for the finalization and dissemination of the 
research findings. 
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The workshop: 


° provided one of the first occasions for countries to compare data collected 
in a standardized way. It showed the importance of having good methods 
and data and justified the WHO strategy of providing standardized 
methods for both national assessment and cross national comparison; 


* permitted the review of the indicators and the political mapping as 
methods of policy analysis and assessments of NDP (field testing) and 
resulted in some improvements which will be incorporated in the next 


edition; 


* allowed countries to share experiences in order to develop better 
approaches for policy formulation and implementation. Although there is 
no standard recipe for improving the policy process in countries, the 
workshop did come up with some useful conclusions; 


e began the process of identifying specific strengths and weaknesses of 
existing policy from national and cross national perspectives; of 
generating hypotheses to explain how different pharmaceutical policies 
can affect the performance of the pharmaceutical sector; and of 
identifying specific policy innovations, 


e showed the necessity to develop such research in countries with different 
socio-economic backgrounds; and to set up sustainable NDP monitoring 
systems in most countries; 


° finally, enhanced the research capabilities of all the participants of the 
12 countries. 


Preliminary results 


The report reviews the main results of the research at this stage in relation to 
(i) the methods used; (ii) the national drug policies; (iii) the cross national 
comparison. These findings are preliminary as not all the country teams had 
finalized the collection and processed the data at the moment of the workshop. 


Standardized indicators and political mapping 


> The aim of the research was to use a set of standardized indicators to 
assess the achievement of the NDP and to use the political mapping 
technique to better analyse the process of policy formulation and 
implementation. During a previous workshop it was agreed that 
countries had the freedom to adapt or modify the indicators to better 
reflect their own country context. A set of common indicators should 
however be preserved given the comparative focus of the project. It was 
also suggested that a common mapping question related to the 
formulation process of the drug policy would probably further the 
process of comparison among countries. However it was decided that, 
given the different requirements of each country, they should be allowed 
to define their own questions for political mapping. 


Executive summary 


. A large percentage of the indicators was useful and did not need 
substantial modification. A few new indicators were constructed by 
country teams to better cover certain components of the pharmaceutical 
policy (e.g. in rational use of drugs). A number of new indicators was 
suggested for specific components or for additional objectives (e.g. 
traditional medicine). The main reasons for not applying certain 
indicators were the absence of data and the inconsistency in data at 
country level. 


° Two countries used the political mapping to better understand what had 
happened in policy formulation and implementation and the reasons for 
successes and failures. One country used the political mapping to assess 
the chances of success of a new strategy to improve the affordability of 
the drugs in the private sector; the other countries looked at specific 
issues closely related to the components which had been assessed in the 
indicators’ exercise. 


* The two methods used in the research were relevant to assess the 
performance of a given national drug policy and the policy process. The 
various categories of indicators achieved their goals. The outcome 
indicators provided a good picture of the national situation in relation to 
the four objectives of the NDP. The structural and process indicators 
allowed the assessment of the level of implementation of the main 
components of the NDP. The indicators revealed new things on the policy 
and illustrated the importance of using certain types of data such as the 
financial data. They were helpful tools for focusing actions on 
deficiencies, and for finding solutions based on data and analysis. The 
method of political mapping helped policy-makers to systematically 
analyse the support and opposition for a proposed policy; consult with 
the major stakeholders on their views; analyse opportunities and 
obstacles to change; design a set of creative and effective strategies for 
change; and assess and track the processes of implementation. 


Strengths and weaknesses of national drug policies 


In most countries covered in the research, the systems/structures /mechanisms 
needed for effective implementation of a national drug policy were in place. 
However, they often did not function properly. Participants were able to identify 
strengths (e.g. established structures, comprehensiveness of the policy, etc.) and 
weaknesses of their country national drug policies (e.g. lack of implementation 
of rules, lack of concerns for financing and pricing issues, etc.). 


In all countries there were components of the NDP which received more 
attention from decision makers and senior management staff; in most of the 
poorest countries these components were essential drugs lists and public 
procurement of drugs under INN. The situation was slightly different in middle 
income countries where the main components varied according to the objectives 
of the policy. Secondly, in all countries some things were easier to do than 
others: put the structures in place, improve the public sector, develop 
components not calling for drastic changes in behaviour, etc. Thirdly, it was 
difficult at this stage of the research to draw conclusions on the relative 
importance of each component in achieving the objectives of the NDP. Finally, 
there were linkages between components and between components and 
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objectives (e.g. absence of quality assurance system > poor quality of drugs, 
withdrawing irrational drugs ™ less irrational use of drugs, etc.). 


In addition, the research clearly demonstrated that policy-making can not simply 
be a technical process but must also be a political process. Policy-makers need 
both technical and political analysis in order to be effective. Policy-making does 
not stop with the official adoption of a policy but continues on through the phase 
of implementation. Some common strategies may exist in national drug policies; 
it is however unlikely that a simple recipe exists for managing the formulation 
and implementation of all national drug policies. The approaches need to be 
developed and refined within specific national political contexts. They need to be 
adapted over time to fit evolving political circumstances and the problems that 
emerge in transforming the policy into practice. 


Lessons from cross national analysis 


The comparative cross national approach pursued in the research was valuable 
as, even at this very preliminary stage of analysis, it was possible to better 
explain the particular aspects of each country’s policy and to identify more 
universal aspects of policy-making. The review of the variations among 
countries led to some tentative explanations, including: 


e There was an obvious positive link between the performance of NDP and 
the economic development, although this did not apply for all the aspects 
of the policy. 

e A number of technical components when implemented adequately made 


a great difference in terms of output: for instance a bad selection of drugs 
and an inefficient procurement system led invariably to shortages of 
drugs; on the contrary, a good registration system had a positive impact 
on rational use of drugs, a local manufacturing industry seemed to have a 
positive influence on prices of drugs as long as the country were big. 


° Countries with no local industry (national or multinational) could 
implement the main aspects of NDP with fewer problems. 


a Compulsory use of an EDL in the public sector was just a first step in 
improving availability and affordability. These objectives seemed more 
difficult to achieve in countries where the pharmaceutical market was 
mostly private and where there was no regulation of drug prices. 


° The geographical situation of countries was also an important factor in 


the difference in performance, e.g.: to be an island was an advantage as it 
made drug imports easier to control. 


° A few motivated people could make a difference between success and 


failure mainly to get things started, but to rely on only a few people was 
dangerous as it did not ensure sustainability. 


lV 
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Improvements of the NDP through radical changes were more likely to 
happen when there were political windows of opportunity. When such 
opportunities presented themselves (e.g. new regimes) it was possible to 
implement quick, radical and widespread changes. When such 
opportunities were not available, it could be better to use a step-by-step 
approach. 


Technical soundness and/or economic rationality of a policy did not 
always imply that the policy was politically viable. 
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Introduction 


|. Introduction 


Pharmaceuticals have made an important contribution to global reductions in 
mortality and morbidity. Making drugs available to the people and ensuring 
they are rationally used is, thus, a priority for every country. However, most 
countries confront a host of problems in their efforts to ensure the availability 
and rational use of safe and effective drugs. In trying to overcome these 
problems, many countries, mainly developing ones, have formulated national 
drug policies (NDPs) that specify national pharmaceutical goals and provide a 
framework for all parties involved. These policies have often been initiated with 
the assistance and active support from the Action Programme on Essential Drugs 
(DAP) of the World Health Organization (WHO), other bilateral and multilateral 
agencies and nongovernmental organizations. 


Although such policies have been implemented by a number of countries, there 
has been no systematic effort to evaluate these policies on either a within- 
country or cross national basis. The project on "Comparative Analysis of 
National Drug Policies" was formulated to fill this lacuna. It was developed 
jointly by the Action Programme on Essential Drugs (DAP/WHO), the 
Department of Public Health Sciences, Division of International Health Care 
Research (IHCAR) at Karolinska Institutet (Sweden) and the Harvard School of 
Public Health (HSPH, Boston). The research took place in Colombia, Guinea, 
India, Philippines, Sri Lanka, Viet Nam, Zambia and Zimbabwe; all these 
countries were part of the original project. An additional four countries decided 
to participate in the research using their own funds (Chad and Thailand) or 
funds provided by the European Union through projects supported by the 
Centre de Recherche et d’Etudes pour le Développement de la Santé (CREDES), 
Paris (Bulgaria and Mali). The present workshop, which took place 10-13 June 
1996 in Geneva, was the third step in the implementation of this collaborative 
project. It followed and built up on a series of meetings, workshops, and country 
work during which the research protocol was developed and the research was 
carried out by the different countries. The research is supported with funds from 
SIDA-SAREC and WHO/DAP. 


This report constitutes five chapters and a series of annexes containing interalia 
data collected in the 12 countries. Following the introduction which provides a 
justification for the need for such research, the second chapter explains the 
objectives of the research, the main research questions and the methods used in 
the project. The third chapter describes the aim and objectives of the second 
workshop with the method of work. The fourth chapter, completed by Annexes 5 
and 6, reviews the preliminary results in terms of the methods used and their 
relevance and outlines lessons learnt in relation to policy development and 
implementation. Finally, the fifth chapter gives the main conclusions and 
recommendations of the workshop. 
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Background on the research project 


lil. Background on the research project 


1. Research questions and objectives of the study 


The overall goal of the research is to assess the performance of national 
pharmaceutical policies - focusing on policy process and policy outputs: 


The principal research question that the project seeks to answer is: 
Have national drug policies (whether they exist as explicit policy 
documents, or are implicit in government actions) been successful in 
achieving the goals of availability, accessibility, good quality, and 


rational use of essential drugs? Why or why not? 


The specific objectives of the research are: 


=> to identify strengths, weaknesses, and political dimensions of 
pharmaceutical policy formulation and implementation within each 
country; 

»> to propose explanations for cross national variations in performance; and 

-»> to propose effective strategies, both national and international, that can 


improve pharmaceutical policy implementation. 


2. Study approach and research tools 


The research provided a cross national, comparative framework. Cross national 
comparison can assist in placing a single country’s experiences within a broader 
international context, and can help explain both the particular features of a 
country’s policy and more universal aspects of policy-making. 


The research combines formative evaluation strategies (focusing on the process 
of policy formulation and implementation) and summative evaluation strategies 
(focusing on policy outcomes). The two main research tools used are standardized 
NDP indicators - for assessment of NDP performance; and political mapping - for 
the analysis of NDP formulation and implementation processes. Group 
evaluation is also part of the methods used. These methods are described more in 
detail in the boxes below and in the research protocol (see Annex 1). 
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Method 1: Standardized NDP indicators 


The indicators are derived from the WHO manual on Indicators for monitoring 
national drug policies (WHO/DAP/94.12). The indicators serve two purposes in 


the research: 


»> to assess the implementation of NDP by measuring progress in key 
components (structural and process indicators), 
> to evaluate the outcomes of NDP (outcome indicators). 


Background Structural Process Outcome 
information indicators indicators indicators 


Quantitative Qualitative Quantitative Quantitative 


Population data Legislation and regulation Availability of 
Economic data Essential drugs selection and drug registration essential drugs 
Health status data Drug allocation in the health budget / public Accessibility of 
Health system data | Sector financing policy peeal drugs 
Public sector procurement procedures 


Public sector distribution and logistics en use of 
rugs 


Quality of drugs 
Human resources 


Drug sector 
organization Pricing policy 
Information and continuing education on drug 
use 


Method 2: Political mapping 
The use of political mapping has three main goals in the research: 


to assess the organizational and political determinants of the policy 
process; 

to help explain the strengths and weaknesses identified by the indicators; 
and 

to assist in identifying and assessing strategies to improve 
pharmaceutical policy implementation. 


As described in the research protocol, the methodology has descriptive, 
explanatory, and prescriptive objectives: to describe the consequences, 
stakeholders, interests, and networks involved in the national drug policy; to 
help explain how and why a particular decision was reached in the past; and to 


assist decision-makers in managing the politics of formulation or 
implementation. 


Background on the research project 


3. Project past activities 


A number of activities took place before this second workshop. They included: 


. A preparatory meeting of the project coordinators in Boston in September 
1993 where a draft research proposal was discussed. It was agreed to 
prepare a background paper which would provide the theoretical 
foundations for the methods to be used in the project. 


. The background paper'reviewed the existing literature on the evaluation of 
social programmes and this review was intended to: provide background 
information on the project's methodological framework for the 
comparative evaluation of pharmaceutical policies in developing 
countries; promote better understanding of key issues related to 
pharmaceutical policy formulation, implementation and evaluation, and 
provide materials that could assist in generating research questions for 
the project. The paper consisted of three main sections: on evaluation 
methods, on problems in the pharmaceutical sector, and on the 
formulation and implementation of pharmaceutical policy. 


* The research protocol was finalized; this document was to be used by each 
country team to prepare its own country protocol (see Annex 2). 


* The background paper and the research protocol were discussed and 
agreed upon by the three collaborating institutions in a meeting in 
Stockholm in March 1994. The countries where the project would be 
implemented were selected, and the formal roles and responsibilities of 
each coordinating institution with regard to the project in general and to 
each participating country in particular were agreed upon. The country 
team leaders were to be identified by the respective collaborating 
institution and to be agreed to by the Ministry of Health of each country. 
The eight countries selected for this project were: Colombia, Guinea, 
India, Philippines, Sri Lanka, Viet Nam, Zambia, and Zimbabwe. These 
countries from Africa, Asia, and Latin America were chosen based on the 
following criteria: socio-economic status, political system and regime 
type, geographical and demographic factors, colonial history, existence or 
absence of a formal NDP, public-private mix and size of domestic 
pharmaceutical industry, availability of in-country contacts, and the 
existence of previous studies. They include some of the world’s poorest 
countries, as well as some middle-income countries; countries with no 
pharmaceutical manufacturing capacity, as well as some with substantial 
capacity. 


* A first workshop was held in October 1994 in Geneva and was the first step 
in the implementation of the research and also the first round of group 
evaluation. Its overall goal was to formally introduce the project to the 
participating countries, initiate work on the individual country research 
protocols, strengthen the research capabilities of country teams by 


Govindaraj, R. Pharmaceuticals and the formulation, implementation, and 
evaluation of drug policies in developing countries, WHO/DAP, May 1994 
(unpublished document). The paper prepared by Dr R. Govindaraj (HSPH) is 
available on request in WHO/DAP. 
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providing training in the research methods to be used as part of the 
project and achieve group consensus on their applications. Research 
methods were presented and discussed thoroughly, draft country 
research proposals, and a plan for the future steps were prepared. The 
workshop was successful in achieving consensus on the methods to be 
used in all studies, thus enhancing the comparative nature of the 
research, leaving the country teams freedom to add to or complement 
these methods’. 


° The research was carried out by the eight countries with ongoing 
technical support from the three collaborating institutions. Some delays 
took place due to problems in transfer of funds, change in principal 
investigator, heavy workload of researchers at country level, etc. All the 
countries carried out the NDP assessment using standardized indicators; 
this work involved in each country a large number of people from 
different sectors and most of the time included national workshops on 
research methodologies and sampling procedures. Five countries out of 

- the eight implemented the political mapping exercise (Colombia, India, 
the Philippines, Sri Lanka and Viet Nam); in India, the Philippines and 
Sri Lanka, the political mapping was undertaken with the direct technical 
support of the HSPH. A research team from Thailand decided to join the 
project and was able to carry out most of the NDP assessment and the 
political mapping. Three additional countries carried out the assessment 
of NDP using indicators: Bulgaria, Chad and Mali (see Table 1). All 
country teams prepared draft reports with preliminary findings and most were 
able to present their findings to policy-makers. 


Table 1: Who has done what? 


Bulgaria 
Chad 
Colombia 


Guinea 
India 

Mali 
Philippines 
Sri Lanka 
Thailand 
Viet Nam 
Zambia 
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Zimbabwe 


Report of a workshop on the research on comparative analysis of national drug 
policies, Geneva, 6-8 October 1994, WHO/DAP (unpublished document). The 
report of the workshop is available on request in WHO/DAP. 


Second workshop 


lll. Second workshop 


The workshop was the second round of group evaluation. It was held in Geneva 
from 10 to 13 June 1996, and organized and supported financially by the WHO 
Action Programme on Essential Drugs (DAP/WHO). 


1. Aim and objectives 


The aim of the workshop was to review the results of the different research 
projects, to solve methodological problems and to prepare a plan for the 
finalization and dissemination of the research findings. 


The specific objectives were: 


Le 


> 


To provide methodological support to the research teams in order to 
improve the research findings. 


To review the results in relation to the overall research objectives, namely: 

« to identify strengths, weaknesses, and political dimensions of 
pharmaceutical policy formulation and implementation within 
each country; 

e to propose explanations for cross national variations in 
performance; and 

e to propose effective strategies, both national and international, 
that can improve pharmaceutical policy implementation. 


To assess the two methods used in the research: standardized indicators 
and political mapping in terms of usefulness, reliability, ease of 
implementation, etc. 


To prepare a plan for the next steps, including the dissemination of the 
findings. 


2. Participants 


The workshop included 26 participants (see List of Participants in Annex 2): 


Researchers from the initial group of countries: Colombia, Guinea, India, 
the Philippines, Sri Lanka, Viet Nam, Zambia, and Zimbabwe. 
Researchers from countries which joined the research later and 
implemented the WHO Indicators for Monitoring National Drug Policies 
and/or the political mapping: Bulgaria, Chad, Mali and Thailand. 
Resource persons to provide technical support in epidemiology and 
statistics. 

Coordinators from the Action Programme on Essential Drugs (WHO), the 
Harvard School of Public Health, and the Karolinska Institutet (IHCAR). 
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Second workshop 


3. Method of work and proceedings 


The workshop was informal with a high degree of participation from country 
teams as well as facilitators. A large amount of preparatory work had been done 
by country teams and facilitators and the discussions reflected this (see Annex 3: 
Agenda). 


The first session of the workshop was devoted to opening remarks by 
Dr F.S. Antezana, Assistant Director General, and by Dr J. Quick, Director of the 
Action Programme on Essential Drugs and to the presentation of the objectives 
and the agenda by Mrs P. Brudon, who was also the main coordinator of the 
research. 


In his presentation, Dr F.S. Antezana pointed out that essential drugs were a 
corner stone of the 1978 Alma Ata declaration on primary health care and that 
the concept of essential drugs reflected the Organization’s basic principle for 
equity. Today, in a time of global change, essential drugs and national drug 
policies were an important aspect of any health sector reform and he hoped that 
the present research project would come up with ideas and strategies on how to 
address changes at global and country levels. 


Dr J. Quick, stressed the fact that 20 years ago only a few countries had national 
drug policies (e.g. Norway, Papua New Guinea, Peru, Sri Lanka). Today more 
than 60 countries have national drug policies and a few others were in the 
process of formulating a national drug policy. The research was therefore very 
timely. It would assist in better understanding how NDPs work and in addition 
it would allow assessment of the two methods used; Dr Quick felt that such 
multicountry research was the right approach for DAP. He believed that an 
important task would be to put the research results into practice and to ensure 
that ministries of health were fully involved in the discussions and 
dissemination of the results. 


The second session was devoted to the evaluation of NDP performance through 
the application of the WHO indicators for monitoring NDP. The session had 
three main objectives: to share the experiences of the countries in carrying out 
the indicators’ assessment, to solve any methodological problems encountered 
by the country teams in applying the indicators, and to discuss the method, the 
findings, and the analysis and interpretation of the findings. 


The session included short presentations by countries on the ways they applied 
the indicators, the results (especially what they have learnt) and the main 
problems encountered. This was followed by comments and questions by two 
discussants and by a general discussion. The main features of these country 
presentations have been included in the relevant parts of Chapter IV. Working 
groups were then established to review methodological problems: one was 
looking at the sampling issues, the second one reviewed the methods to calculate 
a basket of drugs and the third one concentrated on indicators and suggestions 
to improve some of them. The groups assembled again in plenary and presented 
their recommendations. A review of the issues discussed can be found in 
Chapter IV. The last part of the session was allotted to preliminary discussions 
on the relevance of the indicators to assess each NDP, on the findings in terms of 
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policy achievements and on the analysis of the findings to better understand 
policy process. The conclusions of the deliberations are provided in Chapter IV. 


The third session was devoted to political mapping. Each of the six countries 
which undertook political mapping presented its results and the problems 
encountered. This was followed by a group discussion on the results of the 
mapping exercise and its use in assessing policy process. More information is 
provided in Chapter IV. 


The fourth session focused on what could be learnt at this preliminary stage on 
national drug policies through cross national comparisons and on the relevance 
of the two methods for cross national approach. Three working groups reviewed 
the results of each country and began to identify what works and does not work 
across countries, common patterns in policy implementation and factors which 
enable or prevent the formulation and implementation of successful NDP. They 
also assessed the usefulness of the indicators and the political mapping for 
international comparison. More specific comments are included in Chapter IV. 


The fifth and concluding session discussed the next steps: time schedule for the 
preparation of the final country reports, strategies for publication and 
dissemination of the results, funding issues, etc. The main decisions agreed upon 
are summarized in Chapter V. © 


Preliminary findings 


IV. Preliminary findings 


This chapter reviews the main results of the research at this stage in relation to: 
(i) the methods used; (ii) the national drug policies; (iii) the cross national 
comparison and (iv) the broader process of capacity building. It should be noted 
that these findings are preliminary, mainly the ones related to national drug 
policies and cross national comparison, as not all the country teams had finalized 
the collection and processing of the data at the moment of the workshop and 
very few had undertaken a comprehensive analysis of the results. 


1. The methods: What has been learnt? 


Three main issues were discussed during the workshop in relation to the 
methods: their use by the country teams (see 1.1), the methodological problems 
faced by countries in applying them (see 1.2) and the relevance of the methods to 
achieve the objectives of the research (see 1.3). 


1.1 Methods in context: How did the country teams use the 
methods? 


The aim of the research was to use a set of standardized indicators to assess the 
achievement of the NDP and to use the political mapping technique to better 
analyse the process of policy formulation and implementation. During the first 
workshop it was agreed that countries had the freedom to adapt or modify the 
indicators to better reflect their own country context but that as large a set of 
common indicators as possible should be preserved given the comparative focus 
of the project. It was also suggested that a common mapping question related to 
the formulation process of the drug policy would probably further the process of 
comparison among countries; however it was decided that, given the different 
requirements of each country, they should be allowed to define their own 
questions for political mapping. 


Methods in context: the NDP indicators 


All the countries applied the indicators although some did not finalize the data 
collection and analysis before the workshop. 


Four categories of modifications were introduced by countries in the initial list of 
indicators (see also Table 3): 


° Introduction of new indicators: only four countries added new 
indicators; Chad introduced a number of background, structural and 
process indicators already available in the health information system and 
six new indicators related to the level of education of prescribers and 
dispensers; the type of suppliers for the public and the private sectors; the 
number of drugs received in the private compared to the public sector, 
and the percentage of children under five with diarrhoea that have been 
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prescribed ORS. Colombia introduced one additional indicator for 
rational use, namely the percentage of prescriptions dispensed in total by 
the hospital pharmacy. Guinea added eight indicators, four addressed 
more in detail rational use of drugs (WHO/INRUD indicators), the other 
four covered practices related to generics in the private sector. The 
Philippines added indicators on rational use (WHO/INRUD indicators) 
in health facilities and in drug outlets; it also created new indicators to 
compare the practices in the Government and in the private facilities in 
relation to compliance with the EDL and access to a drug formulary. 
Thailand added two indicators related to dispensing practices: percentage 
of drugs dispensed with a drug name on the label (N°1) and with 
instructions for use on the label (N°2). Although the team from Viet Nam 
did not introduce new indicators, they tried to use 24 indicators from the 
proposed ones to assess the implementation of the policy on traditional 
medicine and local production. A number of countries felt that for 
components that were already functioning well, more specific indicators 


may be needed to monitor improvements over time. | 


Modification of existing indicators: three countries changed some of the 
indicators proposed; Chad modified three indicators because of lack of 
data for the past three years. Colombia did the same with three indicators 
to adapt to the national context characterized by a decentralization of key 
functions such as procurement of drugs. Zimbabwe modified 18 
indicators to allow comparisons over time, to simplify calculations, 
because no data were available or to be consistent with previous surveys. 


Withdrawal of indicators because they were not relevant: a number of 
countries decided not to apply some indicators because they felt the 
indicators were not relevant to the country situation. For instance, 
Bulgaria did not apply the structural indicators related to the vehicles of 
the central procurement/distribution unit as such a unit did not exist 
(ST36). Colombia did not apply four structural indicators related to drug 
donations, centralized budget and procurement. The Philippines also 
excluded a few indicators, e.g. the indicator related to the provision of 
foreign currency to the procurement unit as all purchases are done 
internally (ST27). In general, the indicators which were not relevant refer 
to foreign exchange for buying drugs (Colombia, India, the Philippines 
buy internally all the drugs), to centralized structures (Colombia and the 
Philippines are countries where the decentralized process is very 
advanced); to local manufacture (Chad and Guinea have no local 
production), and to collection of drug samples (some countries like 
Zimbabwe have a policy not to collect samples on a routine basis). 


Withdrawal of indicators because data were not available: all countries 
encountered problems in obtaining data. India could not calculate three 
indicators because of the lack of data at national level on drug education 
or the lack of knowledge of the existence of international data (e.g. 
international lists of drug prices). Sri Lanka excluded some indicators 
such as total health expenditure (data not available) and the indicator 
related to banned drugs because the team felt the UN list of banned 
products was not appropriate for the calculation of this indicator. Viet 
Nam could not collect 11 data needed for process indicators on tenders, 
pricing, advertising and banned drugs. Guinea and the Philippines were 
not able to calculate some indicators as some data were not available. 


Preliminary findings 
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Methods in context: the political mapping 
The subjects selected by countries for the political mapping were the following: 


° Colombia: Identifying political statements considered essential for the 
full implementation of the NDP (prospective mapping). 


. India: Explaining the changes in the formulation and implementation of 
the most recent NDP (1994-1995) compared to previous versions of the 
policy (retrospective mapping). 


° Sri Lanka: Assessing the feasibility of getting a new drug pricing formula 
(NPF) enacted. Assisting policy advocates in designing and 
implementing appropriate strategies for the enactment of the NPF 
(prospective and prescriptive mapping). 


° ‘Thailand: Analysing the policy process in the development of the Generic 
Labelling and Advertising Policy (retrospective mapping). | 


° The Philippines: Evaluating the formulation and implementation phases 
of the Generics Act. Understanding the implications of the differences in 
the process and outcomes of these two phases of the policy for the future 
of the Generics Law (retrospective and prospective mapping). 


° Viet Nam: Evaluating the formulation and implementation of the drug 
information policy (retrospective mapping). 
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Methods in context 


NDP indicators: A high percentage of the indicators was relevant and 
useful to the countries situations and did not need substantial 
modifications. A few indicators were constructed by country teams to 
better cover certain components of the pharmaceutical policy (e.g. in 
rational use of drugs). A number of new indicators was suggested during 
the workshop for specific components or for additional objectives (e.g. 
traditional medicine). The main reasons for not applying certain indicators 
were the absence of data and the inconsistency in data. One country felt 
that some data were too sophisticated although useful for pharmaceutical 
systems in developing countries. However, it was agreed that data are 
often available but not easily accessible, mainly financial data which are 
collected in other departments than the pharmacy one. 


Political mapping: Two countries used the political mapping to better 
understand what had happened in policy formulation and implementation 
and the reasons for successes and failures. It should be noted that the two 
countries in question: India and the Philippines have formulated a drug 
policy recently; this policy has very clear objectives and strategies, which 
makes a retrospective analysis easier. One country (Sri Lanka) used the 
political mapping to assess the chances of success of a new strategy to 
improve the affordability of the drugs in the private sector; the other 
countries looked at specific issues closely related to the components which 
had been assessed in the indicators exercise. 


1.2 Methods in question: What were the main problems 
encountered and how to solve them? 


It was the first time for most country teams that they applied the two methods of 
the research; it was also the first time that the two methods were used in such a 
large group of countries and within a comparative framework. It was therefore 
expected by the coordinators that problems mainly related to methodological 
issues such as sampling would arise during the conduct of the research, in spite 
of the training provided during the first workshop. Many of the problems were 
solved underway with the technical support of the coordinating institutions, 
some still remained unsolved for a few teams. This second workshop offered an 
occasion to review the main problems in order to improve the research results 
and to provide suggestions to improve the methods. 


Methods in question: the NDP indicators 
(a) Problems encountered 
Five categories of problems were identified by the research teams 


Methodological problems: A number of countries had difficulties in 
calculating the basket of drugs (India, Mali, the Philippines, Viet Nam) or 


15 


Comparative analysis of NDPs: Second workshop 


———— eee ltl 


16 


(b) 


did not follow the method proposed in the manual (Guinea). The main 
reason was the lack of access to data on disease prevalence and drug 
consumption. It was also felt that the ways to estimate the average price 
of a basket of food could be described in more detail in the manual 
(India). For two countries, structural indicators could be made more 
specific and qualitative (Thailand and Zimbabwe) whereas for one (the 
Philippines) there was a risk of bias depending on the position of the 
researcher for the structural indicators. Also the pharmacy surveys could 
be biased (Sri Lanka) as the pharmacists know that they are surveyed. 


Availability of data: All the teams had problems in obtaining some data, 
mainly the financial ones. In addition data were often incomplete 
(Guinea) or aggregated at levels different than the ones required in the 
indicators (the Philippines, Thailand and Viet Nam). In some cases, the 
reliability of the data was difficult to assess (Guinea, India, Mali, Viet 
Nam and Zimbabwe). 


Collection of data: Some countries faced problems in collecting the data 
because of geographical and logistical difficulties (Chad, Guinea, the 
Philippines), because places to be surveyed were closed (Chad) or 
received too few patients to collect enough prescriptions in two days 
(Guinea). The collection of data is also time consuming as it involves 
visiting many different services (Mali) and training surveyors. “Even 
good training is not a guarantee of the quality of the surveys” (the 
Philippines). 


Data processing and analysis: This can be complicated, entries need to be 
checked before encoding (Guinea, the Philippines). Analyses need to be 
done by knowledgeable people. In addition, another important point was 
to ensure that the chosen indicators really did measure the 
implementation and achievements of the policy. To establish clear links 
between the indicators and the implementation and impact of the policy 
was sometimes difficult (Zimbabwe). Most of the teams emphasized the 
need for a sustainable team to carry out the task regularly and to have the 
process incorporated into the regular activities (Mali, the Philippines, 
Zimbabwe). However, this is not always easy when financial and human 
resources are scarce. 


Data presentation: The presentation of the results to the various target 
groups is not always easy; too many badly explained data do not attract 
attention. It was however not always feasible for the teams to present 
their data in a user friendly format, this was due to lack of time, lack of 
knowledge of some of the opportunities offered by the computers, etc. 


Suggestions to improve the method 


The participants classified the problems they faced during the implementation of 
the indicator exercise in three groups: problems linked with the indicators 
themselves, sampling issues and calculation of the basket of drugs. The 
paragraphs below are the summary of the conclusions of each working group. 


Working group on indicators: The group reviewed all the indicators and 
provided detailed comments on each category of indicators described in 
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the WHO manual. It was emphasized that before any move was made to 
use indicators, it was necessary to ask the question “What do we want to 
measure in relation to our policy? and why?”. 


For the background indicators, the group recommended to use data from 
the International Monetary Fund, the World Bank and national official 
publications as they provide reliability over time and _ facilitate 
comparison between countries. For data not available from these 
organizations, it is important to remember that it is not the “last” decimal 
which matters as the objective of these indicators is to provide the broad 
context and to identify trends. A number of suggestions to facilitate the 
applications of all the indicators were identified and presented at the 
plenary session. 


Working group on sampling: The group reviewed the problems 
associated with sampling in the NDP surveys. It took the cases of the 
Philippines, Thailand, Viet Nam and Zambia as examples. 


>, 


~ Sampling procedures: The sampling procedures varied between the 
countries. In the Philippines, four regions of the country were 
selected. In each region, a sample of private and public facilities 
was drawn from lists available at the Ministry of Health. This 
sample seemed highly representative of the whole population. In 
Viet Nam, the sampling was slightly more complex, with selection 
of 6 provinces (3 in the North, 3 in the South; 2 including the two 
major cities, 2 representing the remote areas, and 2 representing 
the other areas). Again, the sample was most likely representative 
of the population. In Zambia, only two districts were selected, one 
in the North East, and the other in the South East, the most 
populated and economically active of the country. However, 
important parts of the country were omitted. This may introduce 
bias in the final estimates. 


There is obviously a trade off between the representativeness of 
the sample and the cost of the survey, which has to be taken into 
account. The statistical analysis of the results of the survey could 
provide information on the potential bias involved in the 
sampling techniques. 


0, 
~~ 


Choice of secondary units: A critical issue is the choice of the 
secondary units. In general, investigators have stratified their 
samples along the lines indicated in the manual, that is : public 
hospitals (sometimes distinguishing betweert primary, secondary 
and tertiary hospitals), private hospitals, public drug outlets, 
private drug outlets, and remote health facilities. Viet Nam 
selected public hospitals, public drug outlets at district level and 
private drug outlets at provincial level. Investigators noted that 
two other types of secondary units could have been included : 
black markets, and private medical doctors who are allowed to 
sell drugs to their clients. 


% Choice of observation units (data sources): Another critical issue for 
the validity of the studies was the choice of the observation units. 
It was recommended to use a sample of prescriptions. However, 
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in Thailand, investigators working on one of the samples in 
Bangkok had preferred to use “transactions” rather than 
“prescriptions”. A transaction is a real negotiation between a 
client and a drug provider. In some places, transactions without a 
prescription may account for two-thirds of all drug purchases in 
the private drug outlets, even for antibiotics. Drug use in 
“transactions” (roughly self medication) may be quite different 
from drug use in “prescriptions” (normally with proper medical 
advice). 


In one case (Zambia), the investigators added a household survey. 
This was considered important, since the most critical problem in 
this country was access to drugs. This seemed a valuable addition 
to the recommended strategy. 


Other issues: In Thailand, a “surrogate patient survey” was also 
conducted. A surrogate patient is a field worker or research 
assistant who visits health providers and presents them with 
fictitious case scenarios, then records the information. The method 
offers a chance to record non-self-conscious current practice from 
a point of view of the client in a first hand and standardized 
fashion. However, this approach may lead to bias if the field 
worker is not well prepared, and should not be recommended 
unless carefully planned. A participant also noted that this may 
raise serious ethical issues. 


2, 
“~~ 


Working group on basket of drugs: Calculating the basket of drugs was a 
problem for some of the research teams. Bulgaria did it according to the 
manual and were helped by the existing statistical database. The group 
thought that without such a statistical backup (data on prevalence and 
consumption) most countries would find it difficult to calculate the 
basket of drugs. However it was agreed that the method proposed in the 
manual was the best one and should be used where possible. The group 
suggested alternative methods which may be used when the method 
proposed in the manual was not applicable. The alternative methods 
should be submitted to a limited group of experts for validation and if 
relevant should be included accordingly in the new edition of the 
manual. It was also thought that the explanations of the proposed 
method in Part IV of the manual could be improved in terms of clarity. 


Methods in question: the political mapping 


The political mapping analysis was applied in six countries: Colombia, India, the 
Philippines, Sri Lanka, Thailand, and Viet Nam. This lack of complete 
application in all the study countries suggests that problems existed either in the 
training provided for the method, or in the technical support provided by the 
coordinators. In addition, the six country teams encountered some problems 
inherent in the method of political mapping, as a form of applied political 
analysis for examining policy-making processes. The second workshop 
discussed several of these problems and limitations. 
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Preliminary findings 


Problems encountered 


Technical support: The three countries that did not apply the method of 
political mapping had found that they did not have sufficient expertise 
within the country team to learn the new method. In some countries, the 
team may not have included persons with training in policy analysis or 
social science, which usually helps with a political mapping analysis. In 
other countries, the lack of adequate computing facilities may have 
contributed to lack of application. The political mapping method uses a 
Windows-based software program and requires a 486 level processor. 
Technical support from the coordinators could have assisted in applying 
the political mapping method in the three countries that did not complete 
this analysis. 


Quality of data: The political mapping method requires excellent data 
and knowledge about the basic policy-making processes for a specific 
decision. In the six countries where an analysis was performed, these data 
were collected through interviews with stakeholders. An inherent 
limitation of the political mapping method is assuring the quality and 
validity of the qualitative data collected through interviews. For this 
purpose, the study team performs an important role in discussing and 
assessing key judgments about players, their positions, their power, and 
the likely impacts of different political strategies. As with other policy 
analysis methods, political mapping is subject to the limitation of 
"garbage in, garbage out." The quality of the data affects the reliability of 
the analysis. 


Collection of data: The six countries that performed the political 
mapping analysis initially expected some difficulties in the collection of 
data. Most countries, however, did not encounter difficulty in obtaining 
interviews with key decision-makers involved in the policy. Government 
officials and industry executives agreed to participate in the study and 
were, according to country reports, relatively open in discussing the 
policy decisions and the related processes. Accordingly, the collection of 
data was not a major problem for the political mapping analysis. 


Data processing and analysis: The six countries reported a number of 
challenges in carrying out the data processing analysis in the political 
mapping. Several countries noted that the data analysis requires excellent 
knowledge about the policy among team members, and also a "good 
balance" within the team. One way of assessing the reliability of 
qualitative data about policy-making processes is through group 
discussion and collective judgment. For these methods to work well, the 
group needs to be able to function well together and to contain different 
kinds of expertise, both technical and political. Finally, the country teams 
also reported that the method cannot eliminate some unpredictable 
elements in policy-making processes. 


Data presentation: Overall, the six country teams found the computer 
software to be an effective method for presenting the results of the 
political mapping analysis. However, the user still needs to exercise 
judgment in deciding which elements of the analysis should be 
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emphasized and which strategies should be recommended to decision- 
makers. In short, the method of political mapping does not replace 
analytic thinking or policy judgment. Finally, in a rapidly changing 
political environment, it may be necessary to update the analysis and the 
results on a regular basis, otherwise the analysis could be limited by 
specific circumstances. This time-specific dimension is an inherent 
limitation of the political mapping method. 


Suggestions to improve the method 


Technical support: The participants agreed that it would have been 
possible to perform the political mapping analysis, if additional technical 
support had been provided to the three countries that did not carry out 
this part of the research project. Often, an external consultant, working 
with a national expert, can create an effective team of "outsider and 
insider," which is helpful both in collecting and assessing data related to 
policy-making processes. External consultants can also help in 
interviewing some government officials or industry executives, and in 
asking sensitive questions about policy-making processes. 


Training: The participants suggested that additional training on the 
political mapping method, using the updated version of the computer 
software (Policy-maker 2.0) would help with future efforts to perform a 
political mapping analysis. New help texts and tutorials, which are now 
being prepared, are also expected to assist first-time users in applying the 
software and carrying out a political mapping analysis. 


Language: It was also suggested that the political mapping software and 
manual could be translated into French, in order to make the method 
more accessible to French-speaking countries. The current materials are 
available only in English. 


Selection of policy for analysis: The participants suggested that the 
selection of the issues for analysis is a critical factor in carrying out 
political mapping. The analysis seems to work better for well-defined 
issues and policies, in both data collection and data analysis phases. 


Preliminary findings 


Methods in question 


NDP indicators: The main problems faced by research teams were related 
to issues such as sampling, basket of drugs and difficulty to obtain reliable 
data; the teams however agreed that the methods proposed in the WHO 
manual on indicators including the sampling ones were well described and 
practical. They recommended that the suggestions made during the 
workshop be taken into consideration when revising the manual. In 
addition, DAP will provide in a separate document or in the next edition of 


the manual, practical information based on the results of the application of 
the indicators in the 12 countries. 


Political mapping: Main problems encountered by the teams were related 
to the quality of the data (as other policy analysis methods, political 
mapping is subject to the limitation of “garbage in, garbage out”); and the 
data analysis. The data collection was easier than expected and the 
computer software was an effective method for presenting the results. The 
participants suggested additional training on the political mapping 
method. 


1.3 Relevance of the methods 


When defining the research objectives and questions, the coordinators of the 
project selected the WHO set of indicators for monitoring NDP and the political 
mapping as the two best methods for obtaining the answers to the questions 
raised. It was therefore important to review during the workshop the relevance 
of the research: Were the country teams able to assess NDP performance with the 
indicators and the policy process with the political mapping? 


Relevance of the NDP indicators 
The participants reviewed the capacity of the indicators: 


e to assess the level of implementation of the NDP, 
e to evaluate the outcomes of the NDP, 
e to reveal new aspects of the policy, unknown facts and realities. 


A questionnaire was prepared by the facilitators and used by the country teams 
to support this process (see Annex 4). The main preliminary conclusions of the 
workshop can be summarized as follows: 


(a) Assessing the structures and processes of the NDP 


All the country teams found the two categories of indicators (structural and process) 
useful to assess the implementation of the various components of the NDP (see page 4) 
and identify areas that need improvement. Structural indicators were easy to 
collect. Most countries collected between 92% to 100% of them. Countries had 
more problems to calculate process indicators; however, with the exception of 
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countries which had not finalized the study at the moment of the workshop 
(Mali, Zambia) and countries which decided not to collect all the indicators 
(Thailand), countries were able to calculate between 55% (the Philippines) and 
94% of the process indicators (Bulgaria, India and Zimbabwe). For Zimbabwe 
“the quantifiable indicators gave an assessment of the degree and trend of the 
implementation. Further as Zimbabwe had most data available for 1993-1995 it 
strengthened the evaluation of the implementation”. Bulgaria was also able to 
assess the performance of NDP, the same applies for Guinea, India, the 
Philippines and Viet Nam. Colombia could assess the implementation of the 
NDP except in the area of financing of drugs as there was a shift from traditional 
funding to public facilities with fiscal revenues to insurance. For the time being, 
there was not enough available data during the period of transition to properly 
assess this component. Sri Lanka felt also that the indicators work better for 
some areas than for others. Viet Nam stressed the fact that the indicators point 
out the lack of some important activities and strategies which need to be 
developed such as tenders and drug policies, and problems but not the reasons 
for these problems; it is also often difficult to know what is the ideal value of an 
indicator. Finally, Colombia partly supported by Viet Nam and Zimbabwe, felt 
that although the main aspects of the NDP were covered, it was necessary to 
have specific indicators on the implementation of the NDP in the context of the 
health reform. They should measure: 


° The performance of the health authorities (at national, regional 
and local level) in a decentralized context. 
° The performance of the health resources administrators (health 


plans, insurance) to use efficiently the per capita expenditure 
allocated in the system. 


° The structure and process in the management and technological 
development of pharmaceutical services. 
° The performance of the academic sector in function of the 


professionalization and qualification of human resources needed 
in the pharmaceutical sector; and provision of funding and 
technical assistance for the modernization of public sector 
procurement procedures, distribution and logistics (drug 
selection, programming, marketing, reception, storage, 
distribution and dispensing). 

’ The structure and process of the quality of the pharmaceutical care 
services: user's satisfaction, professional competence, security and 
continuity of care. 


(b) Evaluating the outcomes of the NDP 


All the country teams calculated most of the outcome indicators included in the WHO 
manual; Guinea, India and Viet Nam calculated the 10 outcome indicators; the 
Philippines and Sri Lanka could not calculate one indicator, different in each 
case; Thailand was able to collect only a few indicators, however as said earlier 
the Thai team did not perform the indicators study in full. Most countries felt 
that the 10 indicators provided a good measure of what the situation was in 
relation to the four main objectives of a policy, namely availability and 
accessibility to essential drugs and quality and rational use of drugs. For 
Zimbabwe, “the indicators give a good overview and it is easy to include 
additional specific national indicators”. For India “considering the inputs used, 
the results are quite rewarding”. For Chad and Mali the indicators provide 
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“good baseline data which will allow to look at progress”. A few countries felt 
that some additional outcome indicators — quality of pharmaceutical care and 
coverage of the social security system in relation to the total population in 
Colombia, self-medication in the Philippines — would have been useful to 
evaluate better the four objectives: Guinea and Viet Nam which added a few 
indicators on patient care and RUD felt that this “helped to get a more precise 
idea of the outcome of the policy”. 


Most countries would have liked more outcome indicators for additional 
objectives of their policies: traditional medicine was the most frequent missing 
objective (Chad, Guinea, Mali, the Philippines, Thailand and Viet Nam) followed 
by self-reliance and domestic industry (Bulgaria, India, the Philippines and 
Thailand) and by illegal market (Chad, Guinea, Mali, Thailand and Viet Nam). 
Guinea, the Philippines, Viet Nam and Zimbabwe would welcome some 
additional indicators for rational use of drugs at patient and community levels 
including indicators on standard treatment guidelines. 


(c) Discovering new aspects of NDP 


Through the assessment of the performance of the NDP using the different 
categories of indicators all the country teams found things they and their policy- 
makers did not know or expect. For instance, Bulgaria, Chad, Mali, Viet Nam and 
Zimbabwe discovered that the area of drug pricing was not receiving enough 
attention. Colombia “had a general idea about what was going on but the usage 
of the indicators allow them to have a systematic approach to specific 
components of the policy and have a much clearer picture of the present stage in 
the implementation of the NDP”; the same applies for the Philippines, Sri Lanka, 
Viet Nam and Zimbabwe. Guinea discovered that use of drugs was better than 
expected and quality worse, that the cost of a prescription has not changed 
during the last three years at least in the capital, that the average cost of a 
prescription up country was 50% lower than in Conakry and that some 
wholesalers supply also the illegal market. In India it was noted that the increase 
in the price of the basket of drugs was linked to the New Drug Price Control 
Order and that the quality of the drugs in the basket was comparatively poorer than the 
quality of drugs in general; this seems to reflect a shift in the practice of the 
industry, the essential drugs under generic name being left for production to 
small manufacturers, not all of them enforcing GMP. In Mali, it showed that 
prices decreased in the public sector mainly due to a new and improved 
procurement sector. In Sri Lanka the study demonstrated the lack of information and 
data on key areas and specific issues. There was a general belief “that something 
done upstream would have some influence downstream: pharmacies are 
registered and were therefore assumed to be not breaking the law too much. 
Surveys showed that this was not true and the rules were being broken much 
more than it was assumed. The Government has produced a Sri Lanka Hospital 
Formulary and made available sufficient copies for distribution to the prescribers 
in the State Health Sector. However, problems in the distribution system at the 
various levels meant that only a small minority of the prescribers had the 
formulary available with them”. Zimbabwe learnt that the drug policy needs to 
be realistic and more efforts should be devoted to priority setting, that there should be 
a continuous development of the indicators and that the data collection system 
should be simple and sustainable. In Viet Nam, it showed also the lack of 
information and data, the problems of enforcing regulations and the difficulties to 
adapt to market economy. 
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Relevance of the political mapping 


Three main aspects of the political mapping were discussed by the workshop 
participants: the nature and quality of the political mapping analyses that were 
performed, the findings and interpretations of the analyses, and an assessment of 
this method’s ability to achieve the broader objectives of the research project. The 


participants identified the following ways in which the political mapping 
method contributed to an analysis of policy-making processes for national drug 


policies: 


Systematic political picture: The method provided a systematic 
assessment of the political environment in which a national drug policy is 
formulated and implemented. At a minimum, the analysis provided a 
description of the political dimensions of a policy decision, and a method 
to organize the multiple dimensions of a complex decision. One country 
team reported that they had not previously considered the political 
dimensions involved in implementation of a national drug policy. 


Assistance in strategy design: For those countries actively considering 
policy changes and therefore engaged in a prospective political mapping 
analysis (Colombia, the Philippines, Sri Lanka, Thailand and Viet Nam), 
the analysis provided practical assistance in the design of political 
strategies. The systematic political picture and the strategy suggestions 
(included in the software) were reported to be particularly helpful in 
designing strategies by some countries. 


Challenge to assumptions: One team reported that the political mapping 
method was useful in making explicit the team’s assumptions about how 
a new policy would be adopted and in forcing the team to explain and 
justify those assumptions. This process of questioning helped to enhance 
the coherence and feasibility of the policy. The analysis thus allowed the 
team to reflect on some of the unstated assumptions about how a new 
national drug policy would be accepted. 


Reality testing: One team found the interviews to be particularly useful 
in checking the team’s perceptions about other stakeholders. This 
advantage has been reported in other instances of political mapping 
analysis, and is one of the strengths of this method. The analysis helps to 
present a policy decision from multiple perspectives, and helps the 
country team view a problem from the perspectives of other stakeholders. 


Team building: The process of performing the political mapping analysis 
helped create a sense of common language and sense of mission for some 
country teams. The analysis encourages the teams to rethink their 
strategies, taking into account the interactions among policies, players, 
and positions. Finally, the process was thought to strengthen the team’s 
capacity to advocate for a national drug policy. 


Preliminary findings 


Relevance of the methods 


NDP indicators: The participants agreed that the various categories of 
indicators achieve their goals, i.e. the outcome indicators provide a good 
picture of the national situation in relation to the four objectives of the 
NDP, the structural and process indicators allow the assessment of the 
level of implementation of the main components of the NDP; the indicators 
reveal new things on the policy and illustrate the importance of using 
certain types of data such as the financial data; finally, they are helpful 
tools for focusing actions on deficiencies, and for finding solutions based 
on data and analysis. The results of the exercise could also be used to 
achieve political consensus on the national drug policy. Most countries 
recommended that such collection of data should be undertaken regularly 
and budgeted in the MOH budget. There should however be a balance 
between the advantages of having good indicators but which are difficult 
to calculate and the amount of effort and cost needed to collect related 
data. In addition, the workshop recognized that it is difficult to focus on 
monitoring of NDP when the society as such is without "a culture of 
monitoring". The findings and lessons from the analysis are discussed 
below (in sections 2.1 and 2.2). 


Political mapping: Workshop participants agreed that political mapping 
was a new and effective tool that could assist in the analysis of policy- 
making processes and in the design of strategies for both the formulation 
and implementation of national drug processes. While the method has a 
number of inherent limitations and application difficulties (discussed 
above in section 1.2.2), workshop participants believed that they had 
learned important insights from performing the analysis in their country. 
Overall, the participants concluded that the method of political mapping 
helps policy-makers to systematically analyse the support and opposition 
for a proposed policy; consult with the major stakeholders on their views, 
analyse opportunities and obstacles to change; design a set of creative and 
effective strategies for change; and assess and track the processes of 
implementation. The substantive findings and lessons from the analysis are 
discussed below (in section 2.3). 


2. National drug policies: what has been learnt? 


The main aim of the research was to assess if national drug policies (whether 
explicit or implicit) have been successful in achieving availability and 
accessibility to essential drugs, quality and rational use of drugs and to 
understand why and why not. The participants therefore reviewed the data 
obtained from the indicators and the information provided with the political 
mapping. As the data were not completed for all the countries and were not 
available to the coordinators before the workshop, it was not possible to go into 
detail in all these aspects; however, some preliminary analysis and interpretation 
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the next steps of the 
y for the indicators 
during the 


of results could be made which opened exciting avenues for 
research. These preliminary results are presented separatel 
and for the political mapping as they were discussed separately 


workshop. 

2.1 Preliminary findings based on NDP indicators 

To facilitate this discussion, the group used a questionnaire (see Annex 4). 
(a) Assessing the implementation of NDP 


The participants assessed broadly the seven main components of a NDP (see 
Annex 5 and Annex 6), as it is assumed that if these components were 
functioning properly, the objectives would be achieved. For most countries, the 
structures/systems/mechanisms as measured by the structural indicators were in place. 
For example, in all the countries there is a drug regulatory authority whose 
mandate includes registration and inspection (ST4); in all countries except Chad 
and Viet Nam, drugs in the public sector are usually procured through 
competitive tender (ST24). However, the weakest components everywhere are 
the ones related to financing of drugs in the public sector and pricing of drugs in 
the private sector. For example, only in five countries was the public drug 
expenditure per capita more than US$1.00 per year for the last three years 
(ST20). 


In most countries although structures are in place, implementation is not always 
working as expected. As stated by the Guinean team “it is easier to create 
structures than to make them work”. In the same country for example 87% of the 
drugs prescribed in the public sector were on the essential drugs list, the same 
figures from the private sector and the black market were 28% and 39% 
respectively. Zimbabwe provided an opposite example: for the public sector 
procurement procedures, the process indicators were better than the structural 
indicators. Although not all the structures are in place, Zimbabwe obtains very 
good prices for drugs, due to the fact that there are a few very knowledgeable 
and motivated staff at the procurement level. 


Through this assessment, the countries were able to identify areas that need 
improvements, for example Zimbabwe identified the following issues: lead time 
for tenders, private procurement, inspection, information on adverse drug 
reactions, supervision, accounting and finance procedures. 


(b) Achieving the objectives of NDP 


The four objectives of most NDP: availability and accessibility of essential drugs, 
quality and rational use of drugs were assessed using a rating system from 1 to 5 
based on the results of the 10 outcome indicators (see Table 4, Annex 5 and 
Annex 6 which also contain some explanations of the methods used for the 
rating). Availability of essential drugs (OT1 and 2) was quite good in most countries 
with the exception of Zambia. In the case of the Philippines, the availability was 
low mainly in the public sector. As these indicators take into account only a very 
limited basket of the most essential drugs, even a rate like 3is not a very 
satisfactory result. The picture in terms of accessibility to essential drugs is slightly 
less impressive; in many countries the situation mainly in the private sector is not 
good (Bulgaria, Guinea, India, the Philippines and Viet Nam). Quality of drugs 
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remains an important problem in the poorest countries (Chad and Guinea) but is also 
not very good in countries like India, Thailand and Viet Nam. For example, in 
this last country 25% of the samples failed quality control tests. Finally, the 
rational use of drugs is very rare with some countries performing better (the 
Philippines, Zimbabwe). For instance in India 66% of the prescriptions surveyed 
in the public sector contained at least an injection; in Thailand the percentage of 
children receiving antidiarrhoeals from private drugstores was 96%. 


Table 4: Achievement of the objectives of NDP - Preliminary results 


Mali 
Philippines 


Colombia: this figure is for people covered by the social security system 
(17.7 million people out of 37.5 million). 

= Based on data from private drugstores. 

1 = low achievement 

5 = high achievement 


(c) Reviewing strengths and weaknesses of NDP 


Each participant identified the strengths and weaknesses of his/her country 
NDP through a broad analysis of the results of the indicators. It should be stated 
again that it was a preliminary analysis which will be refined once all the data 
will be reviewed and validated. For some countries the main strength they 
discovered was the fact that there are well established structures: legislation 
(Bulgaria, Chad, Colombia, Mali), quality control systems (Viet Nam), 
procurement procedures (Mali), distribution and logistics systems (Bulgaria), 
reliable health data (Bulgaria), well accepted EDL (Mali, Viet Nam); for others, 
the most important strength of the policy was its integration in broader health and 
economic policies (Colombia, Thailand, Zimbabwe). For instance “the main 
strength of the Colombian NDP is its inclusion in the context of the System of 
Social Security established in the country with the health reform. It provides a 
good structure for components related with legislation and regulation, essential 
drugs selection and drug registration, and drug allocation in the health budget. 
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These components of the policy are working quite well in terms of prncese for 
the population already covered by the Social Security System”. For the 
Philippines, the main strengths were the comprehenstveness of the policy involving 
all stakeholders and the fact that the NDP programme 1s officially integrated in 


the Department of Health. 


For many countries, weaknesses of the NDP include lack of implementation of 
rules and regulations (Bulgaria, Chad, Colombia, Guinea, Mali, the Philippines, 
Sri Lanka, Thailand and Viet Nam); no real commitment to monitoring including 
strengthening of data collection (Mali, the Philippines, Sri Lanka, Viet Nam and 
Zimbabwe); not enough concerns for financing and pricing issues (Bulgaria, 
Zimbabwe); lack of financial resources (Chad, Guinea, Mali, Viet Nam); and low 
emphasis on continuing education and training in RUD (Bulgaria, Colombia, 
India, Mali, Sri Lanka and Viet Nam). 


National drug policies: what has been learnt? 


NDP indicators: The results of the indicators, when carefully analysed, 
provided very useful information on the achievements of the policy and its 
main characteristics. Not only did they allow the assessment of the present 
situation but also the identification of areas needing further action and 
areas progressing well. In most countries, the systems/ structures / 
mechanisms were in place; however, they often did not function properly, 
which impeded implementation of strategies and policies and 
achievements of objectives. At this stage of the analysis it is however too 
early to assess the current link between structure and process indicators on 
one side and outcome indicators on the other side, although a careful 
examination of Annex 5 can show evidence of this. Participants were able 
to identify strengths (e.g. established structures, comprehensiveness of the 
policy, etc.) and weaknesses of their country NDP (e.g. lack of 
implementation of rules, lack of concerns for financing and pricing issues, 
etc.). A number of participants felt that the use of the indicators to monitor 
national drug policies should be supported by WHO and by other agencies 
at country level. 


2.2. Preliminary interpretations based on NDP indicators 


A number of hypotheses were made at the beginning of the research on the 
policy process in order to better understand why things work or do not work. 
These hypotheses were based on a review of the literature and on the experience 
acquired by the collaborating institutions in the field of drug policies. The 
hypotheses are related to the implementation of the key components, and cover 
issues such as speed and easiness of implementation; importance of the various 
components and links between components (for more details, see questionnaire 
in Annex 4). The following paragraphs are based on the discussions during the 
workshop and the results of the indicators. 7 


Preliminary findings 


(a) Are there components which have received more attention? 


In all the countries there are components of the NDP which have received more 
attention than others and have been implemented first. For instance, in Chad, 
Guinea, Sri Lanka and Zambia, the essential drugs list, the improvement of the 
drug procurement in the public sector and the strengthening of the regulatory 
aspects (inspection, registration, legislation) have received more support than the 
other components. In Colombia, the development of the pharmaceutical care 
services has been the main purpose of the policy. Priorities to reach this goal 
were set as follows: 


availability of drugs ™ accessibility ® quality ™ rational use 


Therefore, the components which received more attention in Colombia were: 
legislation and regulation; essential drugs selection and drug registration; drug 
allocation in the health budget; pricing policy. 


In India, the pricing issues were the ones receiving more attention because of the 
pressure of the industry and the GATT agreement. In the Philippines, while 
availability and affordability were main objectives of the policy, limited public 
funds and the difficulty of dealing with the economics of pharmaceuticals 
prevent strong action in this field. The quality assurance aspects (strengthening 
of the Bureau of Food and Drugs) received more attention, probably because the 
main actors of the NDP were closely linked to BFAD. Rational use of drugs was 
also the focus of a large number of activities. On the contrary, although only 
drugs on the EDL can be procured in the government sector; nothing really 
substantial has been done in terms of improving public procurement procedures 
including quantification of drugs needs. In Zimbabwe, legislation, essential 
drugs selection, drug distribution in the public sector received priority; this was 
linked to the priorities decided by the National Drug and Therapeutic Policy 
Advisory Committee and to the fact that financial and technical resources were 
available for these activities. Viet Nam focused on improving quality assurance 
systems including registration, legislation and regulations. 


Based on these first findings, it seemed that in most countries the improvement of 
the availability of essential drugs in the public sector through proper selection, good 
procurement and distribution has been given priority. However, two middle income 
countries, India and the Philippines, which have developed very comprehensive 
and different drug policies behaved differently: one has favoured the issue of 
pricing due to the fact that one of the main objectives of the Indian policy is to 
strengthen the domestic pharmaceutical industry; the other has implemented 
first the components related to quality because the main technical people 
involved in the policy were “quality people” and policy-makers and because the 
policy aims to expand the coverage through the use of generic drugs in the 
private sector. 


(b) Are there components which were easy to implement? 


In all countries there are things which are easier to do than others. First the structures 
are relatively easy to set up (EDL, drug regulatory authority, procurement mix) 
but in many cases because of lack of resources, things do not function very well 
(Guinea: “the international aid is willing to support the establishment of central 
medical stores but the operating costs for supervision - fuel ... - are very difficult 
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components related to activities of the private sector are 
(Guinea, Thailand, Viet Nam). Thirdly, components 
behaviour like improving use of drugs are 
also difficult to implement; in addition, they require collaboration between the 
Ministry of Health, the Ministry of Higher Education, district and local health 
authorities, the social security systems when they exist, medical associations and 
the academics (Colombia, Mali, Sri Lanka and Viet Nam). Finally if the policy is 
seeking drastic changes and is implemented on a large scale, it is always difficult 
to think in terms of linear progress, things can work at a moment because of 
strong political support and deteriorate later when the context changes (the 


Philippines). 


to finance”). Secondly, 
less easy to implement 
which involve important changes 1n 


In countries with strong decentralization like Colombia and the Philippines, the 
extent to how easy or difficult it is to implement a component is related mainly 
to the extent of governance of the Ministry of Health over each component and 
its operational capacity for implementing each component in a decentralized 
context. In Colombia, the Ministry of Health has governance for structuring the 
following components: legislation and regulation; essential drugs selection and 
drug registration; drug allocation in the health budget and pricing policy. 
However, because of the decentralization context, implementing those 
components (process) requires participation of district and local health 
authorities. So, the governance is shared heavily with the district and local 
governments, a process which is not easy, due to their poor operational capacity. 


(c) Are there components which have been more important for achieving the 
NDP objectives than others? 


Bulgaria, the Philippines and Thailand felt that all components were important, 
if one component was developed more than another “success will be 
temporary”. Zimbabwe stated that the components should be developed all 
together but for each of them there are basic activities which need to be carried 
out and “you need at that level to make priorities”. In Sri Lanka “since the state 
dominates the health care sector, public sector procurement made a bigger 
impact on achieving the two objectives of essential drugs availability and 
accessibility”. In Guinea it seemed that improvement of diagnosis and 
prescribing practices in the public sector led to a general improvement of use of 
drugs. In Viet Nam the reorganization of the system and the improvement of 
management had positive effects on the drug situation. It was also realized by 
the teams that one cannot expect pharmaceutical systems to be functioning better 


than the health care system as such, although they may in some cases show the 
way (Sri Lanka). 


(d) Are there any linkages in performance across components? 


Although most countries agreed that there are clear linkages between components 
and between components and objectives, the issue was not thoroughly discussed as 
the results have not yet been fully analysed by the teams and the coordinators 
However, a few obvious linkages were noted: absence of quality Riarieiile 
system and low quality of the drugs on the market (Guinea), use of generic drugs 
in the public sector and low cost of treatment compared to what is happenin fe 
the private sector (Chad); withdrawing irrational drugs (no hes pes 
formulation of antidiarrhoeals) meant less irrational use (Sri Lanka); he ube 
sector procurement influenced private sector (Sri Lanka); good ay of fee 
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meant better acceptance of generic prescribing and dispensing (Thailand); using 
tenders meant increased availability of drugs, improving training influenced 
rational use of drugs (Zimbabwe). In Guinea there was no linkage between 
information and continuing education which performed very badly and rational 
use of drugs which was quite good; as said earlier it seems that prescribers in the 
public sector were well trained and influenced practices in the private sector. The 
same remark applied to Zimbabwe where despite continuous training, stock 
management has not improved. This of course lead to questions on the best 
strategies to improve practices and on the content of training. 


National drug policies: preliminary interpretations 


NDP indicators: In all countries there are components of the NDP which 
have received more attention; in most of the poorest countries these 
components are essential drugs lists and public procurement of drugs 
under INN. The situation is slightly different in middle income countries 
where the main components are not the same in all the countries and vary 
according to the objectives of the policy. Secondly, in all countries some 
things were easier to do than others: put the structures in place, improve 
the public sector, develop components which do not call for drastic 
changes in behaviour, etc. Thirdly, it is difficult at this stage of the research 
to draw conclusions on the relative importance of each component in 
achieving the objectives of the NDP. Finally, there are linkages between 
components and between components and objectives (e.g. absence of 
quality assurance system ™ poor quality of drugs, withdrawing irrational 
drugs ™ less irrational use of drugs, etc.). More in-depth analysis is 
however needed to identify reliable trends. 


2.3 Preliminary findings based on the political mapping 


The workshop participants discussed the main findings of the political mapping 
analysis, regarding both the process of the analysis and the substance of the 
analysis. The following preliminary ideas emerged in the discussion. 


(a) The process of political mapping analysis 


Participants agreed that the analysis in six countries demonstrated that it is 
possible to examine policy-making processes, using a systematic step-by-step, 
method, and that the analyses yielded important benefits to both policy-makers 
and policy analysts. This finding supports the proposal that policy-making is not 
simply a technical process, but also a political process, and that policy-makers 
need both technical and political analysis in order to be effective. The method of 
political mapping is one form of applied political analysis, which can assist the 
development and implementation of national drug policies. 
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(b) Policy-making is a continuous process 


Policy-making does not stop with the official adoption of a policy (such as the 
Generics Law in the Philippines), but continues on through the phase of 
implementation. To assure the effective implementation of a policy requires 
continuous tending, caring, revising, and weeding, much like a garden. Lack of 
attention during the phase of implementation can result in a policy falling into 
disrepair or even reversal. In many ways, the most difficult challenges occur 
during implementation, when the goals and mechanisms of a policy must be put 
into practice. 


(c) Interest groups are not often monolithic 


Several country teams reported instances where the ability to split or unite key 
interest groups affected the success or failure of adopting a major change in 
national drug policy. For example, the unification of the domestic and 
international drug industry in India significantly enhanced the chances of the 
major changes in the drug laws in that country. In Thailand as well, the proposal 
of the government united the domestic and international industry, which 
contributed to the reversal of the policy on drug labelling. Whether the 
physicians are unified in a single interest group or divided into specialists versus 
general practitioners can also affect the feasibility of a policy change. Policy 
advocates need to be aware of these potential splits in interest groups, and the 
impact on the feasibility of policy change or policy implementation. Often, these 
splits can be avoided (or created) through effective political strategies and 


“negotiations by policy-makers. 


(d) Policy-making has controllable and uncontrollable events 


Policy-making involves a complex series of strategic choices about the substance 
of a policy, about processes for introduction and adoption, and about 
organizations and mechanisms for implementation. Inevitably, these choices are 
affected by both controllable and uncontrollable events in the local, national, and 
international political environments. Policy-makers can improve the likelihood 
of success of their policies by seeking to identify the controllable events and to 
use them effectively, and be sensitive to the uncontrollable events and present 
their policy appropriately. Strategic concessions in the substance of a policy can 
sometimes improve the chances of success; and the resistance to compromise can 
help create a coalition of opponents. Sometimes, the appeal to higher authorities 
can result in failure, as occurred when Thailand’s FDA sought the opinion of the 
Council of State regarding its drug labelling act. The method of political 
mapping can help policy-makers in identifying both the controllable and 
uncontrollable events, and can suggest strategies for managing both. 


(e) Some common strategies may exist 


Some common strategies may exist that can help promote national drug policies. 
Additional comparative analysis of the six cases of political mapping will be 
necessary to identify these strategies and assess their effectiveness. Nonetheless 
these common strategies would remain as suggestions. It is unlikely that ; 
simple recipe exists for managing the formulation and implementation of all 
national drug policies. The approaches need to be developed and refined within 
specific national political contexts, and adapted over time to fit evolving political 


Preliminary findings 


circumstances and the problems that emerge in transforming the policy into 
practice. 


3. Cross national analysis: What can be learnt at this stage? 


During the last session of the workshop the participants reviewed what could be 
learnt from the comparative nature of the research. They tried to achieve the 
following: 


7 discuss the extent to which the methodologies successfully accomplished 
this cross national comparison; 


a identify and substantiate strengths and weaknesses of existing policy 
from cross national perspectives; 


»> propose explanations for cross national variations in performance; and 


> identify specific policy innovations from cross national learning that can 
improve the implementation of pharmaceutical policies. 


The conclusions presented below are preliminary and need further analysis. 


3.1 Relevance of the methods for cross national comparison 


The group reviewed the usefulness and relevance of the two methods for cross 
national comparison. 


Cross national comparison: the NDP indicators 
The strengths of the indicators for a cross national analysis include: 


® A high percentage of the indicators is relevant and useful in low and 
middle income countries: 80-90% of the indicators have been adopted by 
all countries included in the research. 


e The indicators can be applied in national contexts and measure trends 
when collected over time. 


* Data can be obtained at relatively low cost, compared to the amount of 
resources - public and private - which are spent on drugs. 


e The logical framework (ST/PR/OT) is helpful for the interpretation of 
findings. 


These strengths, associated with the fact that the method provides standardized 
results, make it possible to identify common problems and solutions and to learn 
through the comparison of the findings. The structural, background and 
outcome indicators are more useful for international comparison; for the process 
indicators, targets should normally be set at national level. To facilitate 
comparison, the group tried to introduce a rating system for all types of 
indicators. The same kind of system was used for developing Annex 5. However, 
additional efforts are needed to find optimal ways to relate indicators results to 
the overall scores and to weigh indicators, as all have not the same importance. 
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The group reviewed also some of the limitations of the indicators for cross 


national comparison, which include: 


the indicators are less easy to apply in countries with poor infrastructure 
and lack of reliable data; 


it is sometimes difficult to draw causal conclusions as outcomes are often 
multifactoral; in addition, multiple changes can occur over a single time, 
which makes it difficult to draw strong conclusions about a single factor. 


Cross national comparison: the political mapping 
The strengths of political mapping for cross national comparison include: 


° the development of systematic political analyses, which can be compared 
across countries, to help in assessing the effectiveness of different political 
strategies for the promotion of national drug policies; 


° the in-depth analysis of the policy process, which helps in assessing the 
feasibility of adopting different policy innovations, and the problems of 
implementation for national drug policies. 


3.2 Strengths and weaknesses of NDP from a cross national 
perspective 


A number of the findings which were discussed within each country framework 
are also valid from a cross national perspective. These include that it is easier to 
put structures in place than to ensure they function properly; that nearly everywhere 
availability and affordability are seen as the first objectives to achieve. The participants 
also felt that for certain components there is a universal agreement of their importance 
and of the fact that they should be taken care of everywhere in a kind of 
standardized approach: 


° selection of drugs; 

° procurement by tender in INN; 

° drug pricing policy: governments have a role to play in this area of drug 
affordability although modalities can be different in different countries; 

e continuous education and training is also a universal requirement 


although it is in general the least tackled of all the components. 


In contrast, it seemed difficult to have a standardized pattern for budget 
allocations and public distribution systems. 


It was also acknowledged that although all components are equally important 
everywhere, countries will develop different priorities based on what they perceived 
as their main problems. This will very often depend on the history of the 


pharmaceutical sector and the political and socio-economic characteristics of the 
country. 


Preliminary findings 


Time did not allow a review of the strengths or weaknesses of each country nor 
an attempt to explain them taking as a comparative tool the experiences of the 
other countries. It was agreed that this should be included in the next phases of 
the research. 


3.3 Tentative explanations for cross national variations 


This review of the variations among countries was based on the results of the 
two methods in the various countries. Tentative explanations for these 
differences include: 


* There is an obvious positive link between the performance of NDP and the 
economic development, although this does not apply for all the aspects of 
the policy. 

° A number of technical components when implemented adequately make a great 


difference in terms of output: for instance a bad selection of drugs and an 
inefficient procurement system lead invariably to shortages of drugs; on 
the contrary, a good registration system has a positive impact on rational 
use of drugs (Sri Lanka), a local manufacturing industry seems to have a 
positive influence on prices of drugs as long as the country is big (India). 
It was agreed that this last aspect should be analysed more in detail and 
followed carefully in view of the GATT agreements which may reverse 
the picture. A well-thought drug financing system can influence 
prescribing patterns. However, in all cases, supervision and monitoring 
are necessary. 


* Countries with no local industry (national or multinational) can implement 
the main aspects of NDP with less problems (Sri Lanka). 


° Compulsory use of an EDL in the public sector is just a first step in improving 
availability and affordability. These objectives seem more difficult to achieve 
in countries where pharmaceutical market is mostly private and where 
there is no regulation of drug prices (the Philippines). 


© The geographical situation of countries is also an important factor in the 
difference in performance, e.g.: to be an island is an advantage as it makes 
drug imports easier to control (Sri Lanka). 


‘ A few motivated people can make a difference between success and failure 
mainly to get things started but to rely on only a few people is dangerous 
as it does not ensure sustainability (Zimbabwe). 


3.4 Policy innovations which can improve implementation of 
NDP 


Policy innovations identified by the participants were related to the technical 
components and to the policy process. Under technical components, it seemed for 
instance that the creation of a social security system led to improved outputs of 
the policy (Colombia), the presence of a reimbursement system seemed to 
improve rational use of drugs as did some legislative requirements (e.g. in 
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Sri Lanka, the antidiarrhoeal drugs are forbidden and therefore none of the 
children in the survey received them). 


In terms of policy process, it seemed that improvements of the NDP through 
radical changes are more likely to happen when there were political windows of 
opportunity (e.g. new regimes: the Philippines), changes in the global 
political/economic environment (GATT, global economic liberalization: India), 
etc. When such opportunities present themselves, it is possible to implement 
quick, radical and widespread changes (Philippines). When such opportunities 
are not available, it could be better to use low profile, step by step strategies (Sri 
Lanka). 


Successful implementation of the policy is also influenced by the planning process: 
strategies used for one stage of policy-making affect the probability of success of 
other stages e.g. strategies in the formulation stage affect the probability of 
implementation; therefore planning for implementation is important even at the 
time of the formulation of policy. 


It was stated by the participants that an important lesson of the research at this 
stage was that the technical soundness and/or economic rationality of the policy 
did not always imply that the policy was politically viable. _ 


Cross national approach 


The comparative cross national approach pursued in the research was 
valuable as, even at this very preliminary stage of analysis, it was possible 
to better explain the particular aspects of each country’s policy and to 


identify more universal aspects of policy-making. It also helps in selecting 
policy interventions which have worked in one country and can be 
replicated in others. Finally, the two methods which have been used in the 
project seemed to be useful not only within countries to assess and 
improve the pharmaceutical situation but also for cross national 
comparison and for theory building. 


4. Broader capacity building 
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The research has also allowed an important process of capacity building. 


Although this process has not yet been quantified, all the country teams agreed 
that the research had a number of outputs: 


Preliminary findings 


(a) Pharmaceutical sector benefits 


The results of the research have increased knowledge on key aspects of NDP and 
on successful strategies /policies. This provides an impetus for countries and for 
international agencies to reconsider their own policies in the pharmaceutical 
sector. In addition, the research will be useful in better targeting future research 
in the field of national drug policies, as it will assist in identifying the main 
issues which need to be addressed. 


(b) Benefits for future research 


Conducting the research strengthened skills and research capacity not only of 
the principal investigators but of a large number of people at national level who 
were involved in collection and analysis of data. It also enhanced the ability of 
the researchers to utilize better existing research as the project was an exercise 
where all the participants learnt and benefitted from research conducted in other 
countries. This experienced group of people should play a critical role in the 
future in their own countries. 


(c) Political benefits 


The research improved the information base at national and global level and 
provided evidence to influence policy-makers (Bulgaria, the Philippines) or to 
delay some decisions (Sri Lanka). At the same time, it revealed the lack of 
monitoring and the need to get good data for policy-making. 


(a) Practical tools for policy analysis 


The research project also demonstrated that the two research tools, the indicators 
and the political mapping, could be applied in diverse national contexts, for 
relatively low financial expenditures, and could generate useful information for 
policy-makers. The project indicated that these tools deserved broader adoption 
by policy-makers concerned with national drug policy. 
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Conclusions of the workshop and 
follow-up plans 


V. Conclusions of the workshop and 
follow-up plans 


1. Main outcomes 


The workshop ended with considerable enthusiasm among all participants about 
the results achieved during the few days of the meeting; country teams and 
coordinators felt that the workshop had been a very important step in the 
research and was key for its continuation. 


In summary, the workshop: 


provided one of the first occasions for countries to compare data 
collected in a standardized way. It showed the importance of having 
good methods and data and justified the WHO strategy of providing 
standardized methods for both national assessment and cross national 
comparison, 


permitted the review of the indicators and the political mapping as 
methods of policy analysis and assessments of NDP (field testing) and 
resulted in some improvements which will be incorporated in the next 
edition; 


allowed countries to share experiences in order to develop better 
approaches for policy formulation and implementation. Although 
there is no standard recipe for improving the policy process in 
countries, the workshop did come up with some useful conclusions; 


began the process of identifying specific strengths and weaknesses of 
existing policy from national and cross national perspectives; of 
generating hypotheses _ to explain how different pharmaceutical 
policies can affect the performance of the pharmaceutical sector; and of 
identifying specific policy innovations; 


showed the necessity to develop such research in countries with 
different socio-economic background; and to set up sustainable NDP 
monitoring systems in most countries; 


finally, enhanced the research capabilities of all the participants of the 
12 countries. 
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2. Next steps 


In order to finalize the research and to achieve the total expected outcomes, there 


are still a number of step 


s which need to be carried out in the months to come. 


These are: 


Improvement of the Excel programme DAP has prepared for processing 
and calculating the data for forwarding to the countries (July 1996). 


Finalization of the research at national level (data and reports), to be 
achieved by mid-September 1996. 


Comments from the three coordinating institutions on reports and data 
(end of October 1996). 


National workshops to disseminate the results and test hypotheses (end 
of February 1997). 


Writing of scientific articles and reports by the national teams for 
inclusion in a larger publication (end of April 1997). 


Organization of an international workshop in one of the countries 
involved (India has already proposed to host this). Its objectives will be: 
to review the hypotheses on policy process; to identify specific policy 
innovations from the cross-national approach that could improve the 
design and the implementation of pharmaceutical policy; and to propose 
ways to improve the political feasibility of implementing those 
innovations within specific political contexts. 


Preparation, editing, publishing and dissemination of the results through 
various publications. 


Funding will be sought by DAP and IHCAR to carry out all these activities. If 
funding is not available, a contingency plan will be made at a later stage. 


A number of countries have done the political mapping; for the time being, there 
are no funds to do this exercise in new countries but people are welcome to 
contact Michael Reich for advice by e-mail. It is also possible that students from 


Harvard or people from the region, who are already trained in the methodology, 
can assist countries in the political mapping. 
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I. Introduction 


This research protocol for the Karolinska-Harvard- WHO project on the comparative 
analysis of National Drug Policies (NDPs) in developing countries includes: brief 
description of the problem, the research question, and the study's objectives; the 
anticipated outcomes from the cross-national comparative study; a description of the 
principal analytic methods to be used; and the proposed work plan and project 
collaborators. The accompanying background paper (Govindaraj, 1994) presents a 
comprehensive review of the literature on the formulation, implementation, and 


evaluation of social programs; and provides the rationale and theoretical basis for the 
research protocol. 


II. The Problem 


Effective, safe, affordable, and good quality drugs are an essential element of com- 
prehensive health care (WHO, 1988; Hamrell et al., 1985; Sterky, et al., 1988; London 
School, 1989; Reich, 1987; Kanji, et al., 1992). There is increasing recognition that 
serious problems exist in many current pharmaceutical systems, especially in Third World 
countries (see the Background Paper). The WHO has designed guidelines for the 
development of national drug policies (WHO, 1988) to include legislation/regulation, 
regulatory control, pricing, distribution, drug selection, drug prescribing, dispensing, 
quality assurance, and manpower aspects. The WHO guidelines are based on the concept 
of essential drugs - that a model list of about 250 drugs could serve about 90 per cent of 
the morbidity within a country (WHO, 1977). These guidelines, however, provide little 
information about the processes of policy development and implementation. In addition, 
the WHO is just beginning to develop methods to assess the performance of 
pharmaceutical policy. 


In the past 20 years, numerous initiatives have been taken in formulating and 
implementing pharmaceutical policies in developing countries. During the 1980s, the 
leadership provided in this field by the Action Program on Essential Drugs (APED) at the 
World Health Organization was actively supported by bilateral aid agencies, 
nongovernmental organizations, industry associations, and university research groups. 
Furthermore, many bilateral and multilateral agencies also became involved in promoting 
and supporting pharmaceutical policy development. In the late 1980s, attention turned to 
issues of health financing and drugs (through UNICEF's Bamako Initiative) and to issues 
of drug utilization (illustrated by the International Network on the Rational Use of 
Drugs). In the 1990s, improving the pharmaceutical sector in developing countries was 
cited as a priority by the World Bank in its 1993 World Development Report. 


Various groups have studied national drug policies (NDPs), based on the concept of 
essential drugs (Chowdhury, 1992; Goonaratne, 1989; Kanji, et al., 1989; Kanji, et al., 
1992: Kiatying-Angsulee, et al., 1992; Reich, 1994; Tomson, 1990; WHO, 1992). The 
global policy analysis of the WHO Action Program on Essential Drugs (APED) in 1989 
(London School, 1989) assessed these efforts at pharmaceutical policy development and 
emphasized the spread of Essential Drug Lists as an achievement and the limited 
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adoption of National Drug Policies as a problem. Others identified weaknesses in the 
Information-Education-Communication component together with a lack of focus on the 
rational use of drugs (Finer, et al., 1992). A third group stressed the supply orientation of 
many essential drug programs (Kanji, et al., 1989), and criticized the tendency for 
national drug policy to be reduced to a drug supply system. As pointed out in the 
Background Paper, however, there have been few systematic reviews and analyses of the 
national and international efforts at pharmaceutical policy development and their 
accomplishments. 


The above reports suggest that while the WHO program has effectively identified key 
elements of a national drug policy and has disseminated information on these ideas, 
APED has encountered difficulties in promoting the national implementation of new 
policies and in articulating how the public and private sectors can positively interact in 
the pharmaceutical sector. In addition, the existing studies have suffered from two main 
problems: first, they have lacked systematic comparisons of in-depth national studies; and 
second, they have, therefore, been unable to provide innovative ideas for improving the 
formulation and implementation of national drug policies. This research project seeks to 
address both of these deficiencies. 


III. Research Question and Objectives of Proposed Study 


The overall goal of the project is to assess the performance of national pharmaceutical 
policies in developing countries (focusing on policy outputs and policy process), to 
analyze the reasons for their success or failure, and to propose strategies for 
improvement. The principal research question that the project seeks to answer is: Have 
National Drug Policies (whether they exist as explicit policy documents, or are implicit in 
government actions) in developing countries been successful in achieving the goals of 


availability, accessibility, good quality, and rational use of essential drugs? Why or why 
not? 


The project will combine several disciplines (policy analysis, economics, and 
epidemiology/medicine/pharmacy) in order to assess national accomplishments, and 
develop fresh perspectives on national and international initiatives. Attention will be 
given, where warranted, to the interaction of public and private sectors in the 
pharmaceutical field of each country. Within this context, the specific objectives are: 


a) To identify strengths, weaknesses, and political dimensions of pharmaceutical 
policy formulation and implementation within each country; 


b) To propose explanations for cross-national variations in performance; and 


c) To propose effective strategies, both national and international, that can improve 
pharmaceutical policy implementation. 


Annex 1 


IV. Study Approach 


The project will have three phases: planning, research and dissemination. The research 


phase is described in this section; for details of the planning and dissemination phases, 
see Section 7 below. 


The research phase will evaluate national drug policies in 8 developing countries, and 
will produce a series of country studies that examine key processes of policy formulation 
and implementation. The research will combine summative and formative evaluation 
strategies (see Background Paper). Both quantitative and qualitative measures and 
approaches (described below) will be used in carrying out the analysis. 


The 8 countries selected for this project are: Colombia, Guinea, India, Philippines, Sri 
Lanka, Vietnam, Zambia, and Zimbabwe. These developing countries from Africa, Asia, 
and Latin America have been chosen based on the following criteria: socio-economic 
Status, political system and regime type, geographical and demographic factors, colonial 
history, existence or absence of a formal NDP, public-private mix and size of domestic 
pharmaceutical industry, availability of in-country contacts, and the existence of previous 
studies. They include some of the world's poorest countries, as well as some middle- 
income countries; countries with no pharmaceutical manufacturing capacity, as well as 
some with substantial capacity. 


The implementation of the research phase of the project will involve: 


1: Assessment of NDP performance, through the application of a standard set of 
indicators, in order to evaluate NDP outcomes as well as processes. 


2: Two rounds of group evaluation as a means for a) Achievement of group consensus 
on methodologies, and the training of the country team coordinators (Round 1), and b) 
a comparative analysis of strengths and weaknesses of NDPs, and identifying key 
questions for the political mapping (Round 2). 


3: Analysis of the NDP formulation and implementation processes, through the 
application of political mapping, in order to assess organizational and political 
variables. A second round of political mapping may be undertaken in a subset of the 
study countries, aimed at addressing specific questions that emerge from the first 
round of evaluations and political mapping. 
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V. Anticipated Outcomes 


1. The project will apply a systematic cross-national method for the assessment of 
pharmaceutical policy performance, using summative and formative evaluation 
techniques, through the standardized indicators and the political mapping technique. 


2. The assessment of pharmaceutical policy performance will provide the basis for 
generating hypotheses to explain how different pharmaceutical policies, especially 
different public/private mixes, can affect the performance of the pharmaceutical 
sector. 


3. The project will assist policy-makers responsible for pharmaceutical policy in 
identifying and substantiating specific strengths and weaknesses of existing policy, 
from national and cross-national perspectives, and in formulating strategies to 
improve the implementation of pharmaceutical policy. 


4. The project will provide field testing of the standard indicators and the political 
mapping technique in a series of countries, and their potential development into a 
method of policy analysis and rapid assessment of pharmaceutical sector performance. 


5. The project will facilitate the collection of data for the WHO's forthcoming World 
Drug Situation Report. 


6. The project will identify specific policy innovations from cross-national learning that 
could improve the design and implementation of pharmaceutical policy within each 
country, and will propose ways to improve the political feasibility of implementing 
those innovations within a specific political context. 


7. The project will enhance the capacity of Essential National Health Research within 
each country, through the development and application of the policy performance 


assessment methodology and the political mapping methodology, both of which can 
be applied to other areas of health policy. 


8. The project will provide an impetus for international agencies to reconsider their own 
policies for the pharmaceutical sector, and will assist in strengthening the policy 


analysis capabilities of international agencies involved in supporting the development 
of national pharmaceutical policy. 


VI. Methods 


This project's evaluation of NDPs will use a comparative, cross-national analysis to place 
each country's experiences within a broader international context, and to explain both the 
particular features of a country's policy and more universal aspects of policy-making 
(Reich, 1988). Cross-national studies of health policy in developing countries are 
relatively few in number. In the late 1970s, Montgomery, Lasswell, and Migdal published 
a book on population policies, which used comparative and longitudinal approaches to 
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analyze patterns of policies (Montgomery, et al., 1979). Their study explicitly examined 
both the processes of policy making and implementation as well as the substance of 
population policy, and thereby represents a potential model for this project. A recent book 
on the policies and politics of AIDS in the industrialized countries represents another 
effort to use a comparative approach to health policy from both substantive and 
procedural perspectives (Kirp et al., 1992). There are a few publications of collected 
essays on pharmaceutical problems in developing countries (Robles, et al., 1992; Geest, 
et al., 1988). No studies, however, specifically focus on pharmaceutical policy, and the 
analyses of pharmaceutical policy that exist rarely explore the problems of policy 
performance and implementation in a systematic manner. This project, thus, will 


represent a significant advance in the empirical application of comparative policy analysis 
in developing countries. 


The three methods of standardized indicators, group evaluation, and political mapping are 
described next. 


1. Performance Indicators 


Purpose of Indicators: The standardized indicators represent a tool for summative and 
formative evaluation (see Background Paper). The proposed indicators serve two primary 
purposes: 


a) To evaluate the outcomes of pharmaceutical policies in a comparative framework; and 
b) To assess pharmaceutical policy formulation and implementation (i.e. policy process). 


The outcomes of the NDPs will be measured according to the four common goals of 
NDPs in developing countries: availability, accessibility, good quality and rational use of 
drugs. In addition, other country-specific goals may be added to the evaluation. The 
assessment of success will be based on the extent to which countries have been able to 
achieve the precise goals and targets (initial or evolving) that they set for themselves 
(summative evaluation). In addition, the evaluation will examine the processes of policy 
formulation and implementation, and the institutional and political contexts in which the 
policies were implemented (formative evaluation). 


Selecting the Indicators: The indicators for this comparative evaluation are selected from 
the indicators developed in a recent manual for monitoring national drug policies in 
developing countries (Brudon-Jakobowicz, Rainhorn and Reich, 1994). The indicators are 
classified as general data indicators, structural indicators, process indicators, and outcome 


indicators. 


These are supplemented by some new indicators (in the form of standardized questions) 
based on hypotheses from the Background Paper for assessing relevant aspects of NDP 
formulation and implementation. The draft indicator manual, which lists the indicators, 
their purpose, description, and the methods of data collection, is provided as Appendix 2. 
A summary description of the project's indicators is provided next. 
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Description of Indicators: The policy formulation and implementation indicators 
(standardized questions) are based on hypotheses generated in the Background Paper, and 
focus on the process of pharmaceutical policy formulation. These indicators will 
supplement the political mapping methodology in assessing the organizational and 
political factors that affect the successful formulation and implementation of 
pharmaceutical policy. 


The policy performance indicators are derived from the manual developed by a 
collaboration between APED/WHO, the Harvard School of Public Health, and the Centre 
de Recherches and d'Etudes pour le Développement de la Santé (Paris) (Brudon- 
Jakobowicz, et al., 1994), which followed nine logical steps: 


1. Literature Review to identify potential key issues and components of 
pharmaceutical policy in developing countries. 


2. Delphi Survey to develop consensus on key issues and components of 
pharmaceutical policy in developing countries. 


3. Experts Consultation to review general difficulties in indicator development and 
define criteria for selection of indicators. 


4. Working Group to propose sets of indicators for monitoring implementation of drug 
policy in developing countries. 


5. Field Testing of proposed indicators in six countries to assess the clarity, 
applicability, and usefulness of the indicators selected. 


6. Review of the first draft manual by experts within and outside WHO to assess the 
methodologies used for indicator development and the categories of indicators. 


7. Working group to propose a set of outcome indicators to measure the progress 
toward the overall objectives. 


8. Review of methodology for indicator calculation by some epidemiologists and 
Statisticians within and outside WHO to assess the relevance of the proposed 


methodologies and appropriateness of the sampling procedures. 


9. Finalization of the manual.based on a review of all comments received and 
incorporation of appropriate revisions. 


48 


Annex 1 


The four categories of indicators are briefly described: 


Background information are intended to provide data on the demographic, health, 
economic, and pharmaceutical contexts in which drug policy is being implemented in a 
given country. These indicators are quantitative data, at a single point in time, and in most 
countries are readily available at the central level. Some of the information will be used in 


calculating subsequent indicators. Some data may be useful in cross-national comparisons 
of drug policy implementation. 


Structural indicators provide qualitative information to assess the pharmaceutical system's 
capacity to achieve its policy objectives. These indicators are intended to check if the key 
structures necessary to implement a pharmaceutical policy are present in the country. 
Seven categories of key components are considered: legislation and regulation, essential 
drug selection and drug registration, drug allocation in the health budget/public sector 
financing policies, public sector procurement procedures, public sector distribution and 
logistics, pricing policy, information and continuous education on drug use. The 
indicators can be used in comparing the pharmaceutical policies of different countries and 
in assisting national and international decision-makers to design interventions to improve 
the pharmaceutical sector. 


Process indicators provide quantitative measurements of the processes by which a 
national drug policy is implemented. The indicators assess the degree to which activities 
are being effectively implemented and the progress over time. The process indicator 
monitor the main aspects under the same seven key components of drug policy. These 
indicators can be used for assisting national decision-makers in monitoring progress in 
the implementation of the drug policy in relation to goals and targets set up at the national 
level. 


Outcome indicators are intended to measure the results achieved and the changes that can 
be attributed to the implementation of a national drug policy. These indicators measure 
the effects of the policy on the overall objectives: availability and affordability of 
essential drugs, drug quality, and the rational use of drugs. These indicators can be used 
in assisting national and international decision-makers in measuring progress towards 
overall objectives and adjusting strategies accordingly, and in comparing pharmaceutical 
policies of different countries. 


Implementation of Indicators: The application of these indicators will be carried out by 
the respective country teams. The methodology for calculating the indicator, including a 
detailed discussion of the procedures for conducting surveys, as well as potential sources 
for the required information, are provided in the indicator manual (Appendix 2). 


2. Group Evaluation 


Purpose of Group Evaluation: The group evaluation will be carried out in two rounds, 
with two main purposes: 
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a) To assist in the initiation of the field research by training the participants in the 
indicator and political mapping research methodologies, and to achieve group 
consensus on a common approach [Round 1]. 


b) To review the accomplishments and conclusions of the application of the indicators 
and the political mapping, in a comparative framework, and in a critical but supportive 
group process; and to identify specific questions of possible interest to all developing 
countries implementing NDPs, for further investigation [Round 2]. 


Description of Group Evaluation: Group evaluation will involve the following activities: 
Round 1: 


|) training the. participants in the two research methods and achieving group consensus on 
the application; 


2) comparing the available policy performance indicators for all countries; 
Round 2: 


3) discussing the performance methodologies and proposing adjustments, with particular 
consideration of its potential incorporation into national health policy information 
systems; 


4) discussing cross-national comparisons, and the extent to which the methodologies 
successfully accomplished this objective; 


5) examining proposed explanations for relative strengths and weaknesses of 
pharmaceutical policy performance in different countries; 


6) identifying specific questions relevant to NDP implementation, which could be 
usefully investigated in some countries in a second round of political mapping; and 


7) reviewing policy interventions that could improve pharmaceutical policy performance 
in each participating country. 


Implementation of Group Evaluation: Round 1: The first round of group evaluation will 
occur after each country team has performed a preliminary analysis of the indicators, in 
preparation for the full application of the indicators and political mapping. This meeting 
is scheduled for October, 1994. 


Round 2: The second round of group evaluation will occur after the full application of the 
indicators and the political mapping. Each country team will prepare an evaluation of 
pharmaceutical policy performance, including: 1) the results of the performance 
indicators and the political mapping exercise, 2) an evaluation of the performance 
methodology, and 3) a document that proposes explanations for strengths and weaknesses 
shown by the indicators and the political mapping. These three sets of materials will be 
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critically reviewed by the participants from the three collaborating institutions, and 


representatives from the country teams. Questions to be investigated in certain countries 
will also be identified. 


3. Political Mapping 
Purpose of Mapping: Political mapping has three main goals for this project: 
a) To assess the organizational and political determinants of the policy process; 


b) To help explain the strengths and weaknesses identified by the performance indicators; 
and 


c) To assist in the identifying and assessing strategies to improve pharmaceutical policy 
implementation. 


Description of the political mapping process: Political mapping constitutes the third 
method of the research, and provides an analysis of the process of NDP formulation and 
implementation. The overall objective of political mapping is to assist decision-makers in 
improving the implementation of health policies. 


This methodology has descriptive, explanatory, and prescriptive objectives: to describe 
the consequences, stakeholders, interests, and networks involved in a particular policy; to 
help explain how and why a particular decision was reached in the past; and to assist 
decision-makers in managing the politics of formulation or implementation (Reich, 
1993). 


Political mapping involves six discrete steps, presented in detail in the mapping manual 
(Appendix 3): 


Step One: Policy Consequences: describes the policy's consequences along four 
dimensions: the size, identity, intensity, and timing of the effects. 


Step Two: Position Map: provides a map of the proponents and opponents of the 
health policy, according to international sector, political sector, 
government sector, private sector, nongovernmental sector, and social 
sector. 


Step Three: Stakeholder Analysis: determines the main interests and objectives of each 
organization, and the organizational priority of the interests. 


Step Four:. Policy Network Analysis: identifies the formal and informal linkages 
between organizations and individuals. 
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Step Five: Transitions Assessment: identifies major transitions underway in the 
organization responsible for implementing the policy, the general 
organizational environment, and the broader political environment. 


Step Six: Strategies for Change: analyzes potential strategies for changing or 
influencing policy-making processes and outcomes according to symbolic, 


positive, and negative strategies. 


This methodology is currently under development at the Harvard School of Public Health. 
It has been applied to a series of health policy problems, and has been shown to provide 
rapid identification of problems, to improve communication among organizations, and to 
assist policy-makers in identifying effective new strategies for policy implementation. 


Implementation of Political Mapping: The political mapping will be undertaken by each 
country team, assisted by the three collaborating institutions. The mapping will be based 
on published documents as well as non-published documents (including internal 
government memoranda) and interviews with key policy-makers in the stakeholder 
organizations in both public and private sectors. The mapping process, thus, will require 
active interaction with policy-makers responsible for pharmaceutical policy. It is expected 
that the results of the political mapping will be helpful to policy-makers in designing 
effective strategies to improve implementation. Detailed guidelines on implementing the 
mapping exercise are provided in Appendix 3. 


A second round may be instituted in some countries, in order to pursue questions 
identified by the group evaluation as particularly germane to NDP formulation or 
implementation in developing countries. 


VII. Time Line and Resources 


Project Work Plan 
The project will have three phases: Planning, Research and Dissemination. 


The planning phase will review the conceptual framework and finalize participating 
researchers, including senior people in the international health field who could serve as 
advisors of the project. To assure adequate consideration of private sector issues, advisors 
will also be sought, toward the end of the research phase, from major pharmaceutical 
companies in the United States, Europe, and Japan. A planning session of about 6 to 8 
people was held at the Karolinska Institute in March 1994, to review research outlines 
and analytic approaches. 


The research phase, as mentioned, will produce a series of country studies plus analytic 


essays that compare the performance of national pharmaceutical policies and examine 
policy-making processes at the national and international levels. Particular attention will 
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be given to the mixture of public and private sectors and ways to resolve tensions be- 
tween public and private interests. 


The dissemination phase will begin with an_ international meeting, including 
representatives of the major organizations involved in pharmaceutical policy, with the 
objective of debating the project's research results. Efforts will be made to disseminate 


the conclusions to national policy-makers as well as decision-makers in international 
agencies. 


Project Collaborators 


The three main collaborators in the project are: the Department of International Health 
Care Research (IHCAR), Karolinska Institute; the Department of Population and 
International Health and Takemi Program in International Health, Harvard School of 
Public Health; and the Action Program on Essential Drugs, WHO. The project will be 
coordinated by Dr. Géran Tomson, MD, PhD, of Department of International Health Care 
Research (IHCAR) at the Karolinska Institute, assisted by Dr. Kris Weerasuriya, MD, 
PhD, Senior Lecturer, Department of Pharmacology, University of Colombo, Sri Lanka; 
Professor Michael R. Reich, PhD, of the Department of Population and International 
Health, also Director of the Takemi Program in International Health at Harvard School of 
Public Health, assisted by Dr. Ramesh Govindaraj, MD, MS, Doctoral Candidate, 
Harvard School of Public Health; and Mrs Pascale Brudon-Jakobowicz, Scientist, 
APED, WHO. The coordinators represent the fields of health systems research, political 
science, health economics, and pharmacology. 


Each coordinating institution is responsible for providing support in two or three of the 
eight selected countries: Sri Lanka, Vietnam, and Zambia will be supported by the 
Karolinska Institute; Colombia and India, by the Harvard School of Public Health; and 
Guinea, the Philippines, and Zimbabwe by the WHO. The collaborating institutions will 
be responsible for providing technical support and backstopping in the selected countries. 


Each country research team should ideally include: an epidemiologist/statistician, an 
economist/social scientist, and a policy analyst, and possibly a computer analyst. 
However, it can be difficult in some countries to find such individuals; in this case the 
country coordinator will select the best available researchers. Each country's research 
team will be headed by a coordinator, who will have the overall responsibility for 
carrying out the field research in that country. The country coordinator will also be 
responsible for choosing the other members of the research team, and determining their 
individual responsibilities. The coordinator should have a good knowledge of the 
pharmaceutical sector in his/her country, and should possess the necessary disciplinary 
background, managerial and inter-personal skills, and adequate time to undertake project 


responsibilities. 


The project will begin in Fall, 1993, for a two-year period, with a planning session in 
March. 1994, and a major international meeting to disseminate findings at the end of 
1995. The project schedule is presented below. 
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Project Schedule 


Fall, 1993 


March, 1994 


June, 1994 


July, 1994 


Oct., 1994 


Nov., 1994-Feb., 1995 


March-June, 1995 


July, 1995 


July, 1995 - Nov., 1995 


Dec., 1995 
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- Preparatory meeting among project coordinators, at the 
Harvard School of Public Health 


- Planning session in Stockholm to review the conceptual 
framework and select the participating countries 


- Indicators to be finalized by mid-June. Four background 
documents (background paper, project protocol, 
indicator manual and mapping manual) to be completed 
by mid-June. Country coordinators to be identified by 
the end of June. 


An Official letter of invitation to join the project to be 
sent to each country team, along with copies of 
background documents, etc. by end of July. 


Round | of Group Evaluation: Planning session of 

country coordinators and international coordinators in 
Geneva, to review research methods and initiate research 
project. 


Data collection and analysis of Indicator surveys to be 
undertaken between October, 1994 - Feb., 1995. A 

country visit by a member of the coordinating institution 
will be undertaken in Jan-Feb., 1995. 


Political mapping of the process of NDP 
formulation and implementation to be carried out in 
participating countries 


Round 2 of Group Evaluation: International meeting of 
research team representatives in a developing country to 
discuss research findings and policy implications and 
prepare for next round of mapping 


- Second round of political mapping in selected countries, 
focusing on specific questions 


- Major international meeting to discuss full research 
findings, and disseminate the results 
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Annex 2: List of participants 


Country participants and observers 


Australia 
Unable to attend 


Bulgaria 

Dr Tatjana V. Benisheva-Dimitrova 
National Drug Institute 

Head of Department of Borderline 
Products 

Jarko Sakasov 26 

Sofia 1504 


Tel: 3592 434 7216/7351 
Fax: 3592 442 697 


Dr Guenka Ivanova Petrova 
National Drug Institute 
Department of Borderline Products 
Jarko Sakasov 26 

Sofia 1504 


Tel: 3592 434 7216/7351 
Fax: 3592 442 697 


Chad 

Mr Laurent Ngarmadjingaye 
Ministére de la Santé publique 
Directeur des Etablissements sanitaires 
B.P. 440/1221 

N’Djamena 


Tel: 235 51 55 87/51 76 67 
Fax: 23551 46 26 


Colombia 

Dr Edgar Gonzalez Sedano (new 
position) 

Coordinador Académico 

Programa de Gerencia en Servicios de 
Salud 

Escuela Superior de Administracion 
Publica (ESAP) 

Diagonal 40, N°46A-37 

Santafé de Bogota, DC 


(Former General Director for Health 
Services, Ministry of Health) 


Tel: 571 222 4300/571 213 1575 
Fax: 571 222 4356 
E-mail: egonzal@esap.edu.co 


Guinea 

Dr Mory Fofana 

Ministére de la Santé publique 
B.P. 585 

Conakry 


Tel: 224 464379 
Fax: 224 464 379 


Ms Karin Timmermans 

c/o Dr Kin Shein, EDV 

Regional Office for South-East Asia 
World Health House 

Indraprastha Estate 

Mahatma Gandhi Road 

New Delhi-110002 

Inde 


Tel: 9111331 7804 


Fax: 9111331 8607 
E-mail: karint@who.ernet.in 
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India 
Dr G.N.V. Ramana 
Faculty 


Administrative Staff College of India 


Bella Vista 
Hyderabad, AP500049 


Tel: 9140310952 
Fax: 9140312954 
E-mail: <ramana@asci.uunet.in> 


Mali 
Dr Michel Laloge 


Direction nationale de la Santé publique 


Division Laboratoire Pharmacie 
B.P2tkt 
Bamako . 


Tele 22> 25 24.63 
Fax: 223 23 2463 


Mr Ousmane Sidibe 

Cellule Planification statistique 
Chef Division statistique 
M53... 

ERY. 252 

Bamako 


Tel: 225 23 27 25 
Faxes ~~ 223 23 02 03 


Philippines 

Dr Edelisa Carandang 

National Drug Policy 
Department of Health 

Bureau of Food and Drugs Bldg. 
DOH Compound, Alabang © 
Muntinlupa, Metro Manila 


Tel: 632 807 0751/807 0721 
Fax: 6328070751 


E-mail: <edcarrud@webquest.com> 


Russia 
Unable to attend 
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Sri Lanka 

Dr Krisantha Weerasuriya 
Department of Pharmacology 
Medical Faculty 

University of Colombo 

P.O. Box 271 

Colombo 


Tel: 941695230 
Fax: Oat 578 170 
E-mail: <kw_twcp@slt.Ik> 


Thailand 

Dr Sauwakon Ratanawijitrasin 
Faculty of Pharmacy 
Chulalongkorn University 
Phayathai Road 

Bangkok 10330 


Tel: 6622511871 
Fax: "O62 20 002 27 
E-mail: <fphasrt@chulkn.car.chula.ac.th> 


Viet Nam 

Dr Nguyen Thanh Do 

Director and Head 

Department of Social Pharmacy and 
Drug Information Centre 

Ha Noi College of Pharmacy 

13-15 Le Thang Tong St. 

Hanoi 


Tel: 844 824 8703 
Fax: 844-825 1879 


Zambia 

Dr Ndashi W. Chitalu 

Department of Community Medicine 
University Teaching Hospital 

P.O. Box 50110 

Lusaka 


Tel: 260 1 294755 (Home) 
Fax: 260 1 294755 


Mr Ronald M. Kampamba 

Department of Pharmaceutical Services 
Ministry of Health 

P.O. Box 31890 

Lusaka 
Tel: 260 1 229021 

Mr Mbiko Msoni 

Institute for African Studies 
University of Zambia 

P.O. Box 30900 

Lusaka 


Tel: 2601 294131 


Resource persons 


Mrs Pascale Brudon 

Scientist 

Action Programme on Essential Drugs 
CH-1211 Geneva 27 


Tel: 4122791 3512 
Fax: 4122791 41 67 (DAP fax) 
E-mail: <brudonp@who.ch> 


Dr Torkel Falkenberg 

Karolinska Institutet 

Department of International Health 
and Social Medicine 

IHCAR 

S-17177 Stockholm 


Tel: 468 7286928 
Fax: 468311590 
E-mail: <torkel.falkenberg@fa.ki.se> 


Annex 2 


Zimbabwe 

Mr Misheck Ndlovu 
Masvingo Medical Stores 
P.O. Box 520 

Masvingo 


Tel: 
Fax: 


263 39 62428 /64169 
263 39 62460 


Mrs Birna Trap 

Zimbabwe Essential Drugs 
Action Programme 
Ministry of Health 

P.O. Box CY 924 
Causeway 

Harare 


Tel: 2634795 353 

Fax: 2634795 353 

E-mail: <btrap@healthnet.zw> 
Healthnet: 5: 7211/2.2973 


Dr Michel Garenne 

Directeur de Recherche 

Centre francais sur la Population et le 
Développement 

15, rue de l’Ecole de Médecine 
F-75270 Paris Cedex 06 


Tel: 3314441 82 34 
Fax: 33144418231 
E-mail: <garenne@ceped.ined.fr> 


Dr Ramesh Govindaraj 

c/o Takemi Program in International 
Health 

Harvard School of Public Health 

665 Huntington Avenue 

Boston, Massachusetts 02115 

USA 


Tel: 1617 432 0361 


Fax: 1617 4321251 
E-mail: <rgovinda@hsph.harvard.edu> 
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Dr Jean-Daniel Rainhorn 
Directeur 

CREDES 

53, rue de Turbigo 
F-75003 Paris 


Tel: 33142741110 
Fax: 331427411 20 
E-mail: <101350.1011@compuserve.com> 


Dr Anne V. Reeler (new position) 
Technical Officer 

UNAIDS 

CH-1211 Geneva 27 


Tel: . 4122791 4742 
Fax: 4122791 4187 
E-mail: <reelera@unaids.org> 


Professor Michael Reich 

c/o Takemi Program in International 
Health 

Harvard School of Public Health 

665 Huntington Avenue 

Boston, Massachusetts 02115 

USA 


Tel: 1617 432 0686 
Fax: 16174321251 
E-mail: <reich@hsph.harvard.edu> 


Dr Goran Tomson 

Karolinska Institutet 

Department of International Health 
and Social Medicine 

IHCAR 

S-17177 Stockholm 


Tel: 468 7286928 
Fax: 468 311590 
E-mail: <goran.tomson@ihc.ki.se> 
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Annex 3: Agenda 


Second workshop on the research on the comparative analysis 
of NDP 


Action Programme on Essential Drugs 
World Health Organization 


Room M-505 


Day 1, Monday, 10 June 1996 


Session I: Introduction 


09:00-10:00 Introductory remarks by Dr Fernando S. 
Antezana, ADG 


Opening of the workshop by Dr Jonathan D. 
Quick, Director, DAP 

Introduction of participants 

Introduction to the workshop by Mrs Pascale 
Brudon, DAP 


Session II: | NDP Performance assessment (indicators) 


10:00-16:00 Presentation by countries (15 minutes each) 
Presentation by countries (15 minutes each) 


16:00-17:30 Identification and discussion of key 
methodological problems 


Day 2, Tuesday, 11 June 1996 


09:00-11:00 Discussion and resolution of key 
methodological problems 


11:00-16:30 Analysis and interpretation of the findings 


16:30-17:30 Synthesis of Session II 
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Day 3, Wednesday, 12 June 1996 


Session III: Political mapping 


09:00-12:00 Presentation by countries 


14:00-16:00 Discussion on similarities and differences 


16:00-17:00 Plenary and synthesis of Day 3 


Day 4, Thursday, 13 June 1996 


Session IV: 


09:00-12:30 Cross national comparison: 

® relevance of the systematic cross national 
method which has been used in the research for 
assessing NDP; 

é review of the hypothesis developed to explain 
how different strategies affect the outcomes of 
the policy; 

® review of the explanations developed for cross 
national variations in NDP performance. 

e formulation of national and_ international 


strategies and specific policy innovations to 
improve implementation of national drug 
policies. 


14:00-16:00 Planning next steps 


° time schedule for the preparation of the final 
| reports, including methodological refinements; 
® Strategies for dissemination of the results, 


including national and international seminars 
and publications; 


e collaboration with policy-makers and donor 
organizations; 
2 funding issues. etc. 
16:00 Closing remarks 
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Annex 4: Questionnaire on NDP 
performance assessment 


Second workshop on the research on the comparative analysis 
of NDP 


1. The relevance of the method to assess your country’s policy 


is Were you able through the use of the indicators to evaluate the 
outcomes of your NDP? 


i- Were you able through the use of the indicators to assess the 
implementation of the NDP? 
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3. Did you find things you did not know about the NDP from the 


indicators? 


ee ee eee 
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4. Which aspects of the NDP were not covered by the indicators? 
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a The findings: 
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Based on the data collected (ST, PR, OT indicators), there are at least four 
questions: 


Please circle the correct number. 
1 = does not work 
5 = works very well 


1. Has the policy achieved the 
objectives of: | 


e availability of essential drugs? i 2 3 3. Me 
e accessibility to essential drugs? il 2 ? 4 a 
e quality of drugs? 1 2 3 4 2 
e rational use of drugs? 1 2 3 4 > 


y B 
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Which components of the policy work and which ones do not 


work (policy structures and processes)? 


Please circle the correct number. 
1 = does not work 
5 = works very well 


(a) 


(b) 


(c) 


(d) 


(e) 


(f) 


3. 


Legislation and regulation 
Essential drugs selection and 
drug registration 

Drug allocation in the health 
budget/public sector 
financing policy 

Public sector procurement 


procedures 


Public sector distribution and 
logistics 


Pricing policy 
Information and continuing 


education on drug use 


Comments, if needed: 


What is your assessment of the strengths of the national drug 


policy? 


nn 


a ——— 
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4. What is your assessment of the weaknesses of the national drug 


policy? 


a The analysis and interpretations of the findings 


66 


Yes No 
13 Are there components (a, b, c, d, e, ...) which have J} | 
received more attention and were implemented 
first? If yes, can you identify the reasons for this? 
ys Were there components (a, b, c, d, e, ...) that were [] I 


easy to implement and components that were 
difficult to implement? If yes, can you identify 
which ones and why? 


Annex 4 


3. Are there components (a, b, c, d, e, ...) which seem _} L} 
to have been more important for achieving the NDP 
objectives than others? If yes, can you explain why? 


4. Were there any linkages in performance across [_] LJ 
components, so that if one component was 

performing well, other also performed well? If yes, 

can you comment upon? 
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Annex 5: Achievements of the national 
drug policies in the 12 countries 


This annex contains a one-page table for each country which used the NDP 
indicators for assessing its national drug policy, namely Bulgaria, Chad, 
Colombia, Guinea, India, Mali, Philippines, Sri Lanka, Thailand, Viet Nam, 
Zambia and Zimbabwe. 


Each page contains rated information on: 


1. The achievement of the NDP in relation to the four objectives: availability 
of essential drugs, accessibility of essential drugs, quality of drugs, 
rational use of drugs. 


The level of implementation of each key component of the NDP: 
legislation and regulation, essential drugs selection and drug registration, 
drug allocation in the health budget/public sector financing policy, 
public sector procurement procedures, public sector distribution and 
logistics, pricing policy, information and continuing education on drug 
use (structure and process). 


The rating was done by the teams during the workshop on the basis of the data 
collected in each country. It used both calculation of averages and judgment 
related to the relative importance of the various indicators in each subgroup. It 
was refined later in DAP once new and more accurate data were sent by the 
countries (see Annex 6). 


It should however be seen as tentative and is provided in this annex as an 
example of what can be achieved with the results of an indicator exercise. 
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Achievement of NDP 
BULGARIA 
1. Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 3 4) 9 
e accessibility to essential drugs? 1 2) 3 4 5 
e quality of drugs? 1 2 wy 4 5 
e rational use of drugs? 1 2 © 4 9 
a Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 p 3 4) 5 
Process 1 ee 3 4 5 
b. Essential drugs selection Structure 1 2 «} 4 5 
and drug registration 
Process 1 2 3) 4 S 
c. Drug allocation in the Structure 1 eo 3 4 5 
health budget/ public 
sector financing policy Process 1 2) 3 4 9 
d. Public sector procurement Structure 1 2 3 4 @ 
procedures 
Process 1 2 3 4 3 
e. Public sector distribution Structure 1 2 3 4) 5 
and logistics 
Process 1 2 3 4) 5 
f. Pricing policy Structure 1 2: © 4 5 
Process 1 2 &) 4 5 
g. Information and continuing Structure 1 2 3] 4 5 
education on drug use 
Process 1 2 3 4 9 


The correct number is circled. 
1 = does not work 
5 = works very well 
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Achievement of NDP 
CHAD 
Ue Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 © 4 9 
e accessibility to essential drugs? 1 2 3] 4 5 
e quality of drugs? 1 2 3 4 o 
e rational use of drugs? 1 2 &) 4 5 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2) 3 4 5 
Process @ 2 3 4 5 
b. Essential drugs selection Structure 1 @ 3 4 & 
and drug registration 
Process 1 2) 3 4 5 
c. Drug allocation in the Structure 1 2 © 4 5 
health budget/public 
sector financing policy Process 1 2 3 4 cs 
d. Public sector procurement Structure 1 2) 3 4 5 
procedures 
Process 1 2 3 4 5 
e. Public sector distribution Structure 1 &2 3 4 5 
and logistics 
Process 1 2 © 4 5 
f. Pricing policy Structure 1 2) 3 4 5 
Process 1 (2) 3 4 5 
g. Information and continuing Structure 1) 2 3 4 5 
education on drug use 
Process 1 id 3 4 § 


The correct number is circled. 
1 = does not work 
5 = works very well 
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Achievement of NDP 
COLOMBIA 
1S Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 3 4) 9 
e accessibility to essential drugs? 1 2. 3 4) 9 
e quality of drugs? 1 2 3 4) 9 
e rational use of drugs? 1 2 © 4 5 
2: Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2 3 4) 5 
Process 4p 2 3 4 5 
b. Essential drugs selection Structure 1 2 3 4) 5 
and drug registration 
Process 1 e 3 4 | 9 
c. Drug allocation in the Structure 1 2 3 4) 5 
health budget/ public 
sector financing policy Process 1 2 &} 4 5 
d. Public sector procurement Structure 1 2 3 4 @ 
procedures 
Process 1 e 3 4 9 
e. Public sector distribution Structure 1 2 3 4) 5 
and logistics 
Process 1 2 3 4) 9 
f. Pricing policy Structure 1 2 3 4) 5 
Process 1 2 &) 4 5 
g. Information and continuing Structure 1 2) 3 4 5 
education on drug use 
Process 1 2) 3 4 5 


The correct number is circled. 
1 = does not work 
5 = works very well 
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Achievement of NDP 
GUINEA 
1. Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 3 4] 5 
e accessibility to.essential drugs? 1 Q 3 4) SS) 
e quality of drugs? 1 2] 3 4 5 
e rational use of drugs? 1 2 (3) 4 5 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2 3 4) 5 
Process as 2 3 4 5 
b. Essential drugs selection Structure 1 2 3 4) 5 
and drug registration 
Process 1 2 © 4 Ss) 
c. Drug allocation in the Structure 1 ®@ 3 4 5 
health budget/public 
sector financing policy Process 1 2) 3 A 5 
d. Public sector procurement Structure 1 2 «) 4 5 
procedures 
Process 1 2 €) 5 
e. Public sector distribution Structure 1 2 3 4) 5 
and logistics 
Process 1 2 3 4) 5 
f. Pricing policy Structure 1 2) 3 4 5 
Process 1 2 © 4 5 
g. Information and continuing Structure ae 2 3 4 5 
education on drug use 
Process a 2 3 4 5 


The correct number is circled. 
1 = does not work 

5 = works very well 

* for the private sector 
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Achievement of NDP 
INDIA 
i. Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 © 4 ° 
e accessibility to essential drugs? 1 & 3 4 5 
e quality of drugs? 1 vt 3 4 5 
e rational use of drugs? 1) 2 3 4 5 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 “3 3 4) 
Process é) 2 3 4 
b. Essential drugs selection Structure 1 2 3 4) 
and drug registration 
Process 1 2 © 4 
c. Drug allocation in the Structure 1 2) 3 4) 
health budget/ public 
sector financing policy Process 1 2) 3 4 
d. Public sector procurement Structure 1 2 3) 4 
procedures 
Process 1 e © 4 
e. Public sector distribution Structure 1 2 3 4) 
and logistics 
Process 1 2 3 4 
f. Pricing policy Structure 1 2] 3 4 
Process 1 2 &) 4 
g. Information and continuing Structure 1) 2 3 4 
education on drug use 
Process @ “ 3 ' 


The correct number is circled. 


1 = does not work 
5 = works very well 
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Achievement of NDP 
MALI 
: Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 © 4 5 
e accessibility to essential drugs? 1 2 3 4 5 
e quality of drugs? 1 2 3 4 9 
e rational use of drugs? 1 2] 3 4 5 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2 3 4) 5 
Process 1 ® 3 4 5 
b. Essential drugs selection Structure 1 2 3 4) 5 
and drug registration 
Process 1 @ 3 4 5 
c. Drug allocation in the Structure 1 2) 3 4 5 
health budget/public 
sector financing policy Process 1 2 3 4 5 
d. Public sector procurement Structure 1 2 3 4] 5 
procedures 
Process 1 2) 3 4 5 
e. Public sector distribution Structure 1 2 (3) 4 5 
and logistics 
Process 1 2] 3 4 5 
f. Pricing policy Structure 1 2 © 4 5 
Process 1 2 & 4 5 
g. Information and continuing Structure 1 Le) S 4 5 
education on drug use 
Process iP 2 3 4 5 


The correct number is circled. 
1 = does not work 
5 = works very well 
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Achievement of NDP 
PHILIPPINES 
1. Has the policy achieved the objectives of: 
¢ availability of essential drugs? 1 2 © 4 9 
e accessibility to essential drugs? 1 2 © 4 9 
e quality of drugs? 1 2 3 4) 5 
e rational use of drugs? 1 2 3 4 | 9 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2 3 4) a 
Process 1 2 &} 4 5 
b. Essential drugs selection Structure 1 2 3 4) 5 
and drug registration 
Process 1 c 3 4 | 9 
c. Drug allocation in the Structure 1 2) 3 4 5 
health budget/ public 
sector financing policy Process 1 2 3 4 § 
d. Public sector procurement Structure 1 2 &} 4 5 
procedures 
Process 1 2 3 4 5 
e. Public sector distribution Structure 1 2 3 4) 5 
and logistics 
Process 1 2) 3 4 9 
f. Pricing policy Structure 1 2 © 4 9 
Process 1 2 3 4 5 
g. Information and continuing Structure 1 2 3 4) 9 
education on drug use 
Process 1 2 3] 4 5 


The correct number is circled. 
1 = does not work 
5 = works very well 
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nr 


Achievement of NDP 
SRI LANKA 
8 Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 3 4 © 
e accessibility to essential drugs? 1 2 3 4) 5 
e quality of drugs? 1 2 © 4 9 
e rational use of drugs? 1 2 © 4 9 
2: Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2 © 4 5 
Process 1 2 (3) 4 5 
b. Essential drugs selection Structure 1 2 | 4) 5 
and drug registration 
Process 1 2 © 4 SS) 
c. Drug allocation in the Structure 1 2 3 4) 5 
health budget/public 
sector financing policy Process 1 2 3 4) 5 
d. Public sector procurement Structure 1 2 3 4) 5 
procedures 
Process 1 2 3 4 5 
e. Public sector distribution Structure 1 2) 3 4 5 
and logistics 
Process 1 2 2 4 5 
f. Pricing policy Structure 1 2 2) 4 5 
Process 1 2 3 a 5 
g. Information and continuing Structure 1 2 © 4 5 
education on drug use 
Process 1 2) 3 4 5 


The correct number is circled. 
1 = does not work 
5 = works very well 
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Achievement of NDP 
THAILAND 
1. Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 3 4 <) 
e accessibility to essential drugs? 1 2 3 a 5 
e quality of drugs? 1 @ 3 4 5 
e rational use of drugs? Q* 2 3 4 5 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2: 3 4) 5 
Process 1 2 &. 4 5 
b. Essential drugs selection Structure 1 2 3 4) 5 
and drug registration 
Process 1 2) 3 4 5 
c. Drug allocation in the Structure 1 2 3 4 @ 
health budget / public 
sector financing policy Process 1 2 3 4 5 
d. Public sector procurement Structure 1 2 &} 4 5 
procedures 
Process 1 2 3 4 5 
e. Public sector distribution Structure 1 2 &) 4 5 
and logistics 
Process 1 2 3 o 
f. Pricing policy Structure ag 2 3 4 5 
Process 1 2 3 4 ) 
g. Information and continuing Structure 1 2 3 4 5) 
education on drug use 
Process 1 e 3 4 <) 


The correct number is circled. 

1 = does not work 

5 = works very well 

* mainly based on data from private drugstores 
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Achievement of NDP 
VIET NAM 
> Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 3 4 9 
e accessibility to essential drugs? 1 2 © 4 9 
e quality of drugs? 1 2) 3 4 9 
e rational use of drugs? 1 2) 3 4 S 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2 © 4 5 
Process 1 2 3) 4 5 
b. Essential drugs selection Structure 1 2 &) 4 5 
and drug registration 
Process 1 2) 3 4 <) 
c. Drug allocation in the Structure 1 2 3) 4 & 
health budget/ public 
sector financing policy Process 1 2 (3) 4 5 
d. Public sector procurement Structure 1 2) 3 4 5 
procedures 
Process 1 2 3 4 5 
e. Public sector distribution Structure 1 2 ©) 4 5 
and logistics 
Process 1 2 3 4) ss) 
f. Pricing policy Structure ub 2 3 4 § 
Process 1 2 (3) 4 5 
g. Information and continuing Structure 1 2. 3 (4) 5 
education on drug use 
Process 1 2 © 4 ) 


The correct number is circled. 
1= does not work 
5 = works very well 
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Achievement of NDP 
ZAMBIA 
7 Has the policy achieved the objectives of: 
e availability of essential drugs? 3 4 <) 
e accessibility to essential drugs? 3 2 5 
e quality of drugs? 3 ~ 5 
e rational use of drugs? 3 = 5 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 3] 4 SS) 
Process 3 4 5 
b. Essential drugs selection Structure © a 5 
and drug registration 
Process 3 4 9 
c. Drug allocation in the Structure 3 “ 5 
health budget/ public 
sector financing policy Process 3 4 5 
d. Public sector procurement Structure &} 4 5 
procedures 
Process 3 4 9 
e. Public sector distribution Structure &} 4 5 
and logistics 
Process aa 4 5 
f. Pricing policy Structure 3 4 5 
Process 3 4 5 
g. Information and continuing Structure 3 4 5 
education on drug use 
Process 3 4 5 


The correct number is circled. 
1 = does not work 
5 = works very well 
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Achievement of NDP 
ZIMBABWE 
1. Has the policy achieved the objectives of: 
e availability of essential drugs? 1 2 3 4) 5 
e accessibility to essential drugs? 1 2 3 4) 5 
e quality of drugs? 1 2 &) 4 5 
e rational use of drugs? 1 2 3 4) 5 
2. Which components of the policy work and which ones do not work 
(policy structures and processes)? 
a. Legislation and regulation Structure 1 2 3 4) 5 
Process 1 2) 3 4 5 
b. Essential drugs selection Structure 1 4 3 4 15) 
and drug registration 
Process 1 2 3 4) Ss) 
c. Drug allocation in the Structure 1 2 © 4 5 
health budget /public 
sector financing policy Process 1 2 3 4) 5 
d. Public sector procurement = Structure 1 2 &) 4 5 
procedures 
Process 1 2 3 4) 5 
e. Public sector distribution Structure 1 2 3 4) 5 
and logistics 
Process 1 2 © 4 5 
f. Pricing policy Structure 1 2] 3 4 5 
Process 1 2 3 (4) 5 
g. Information and continuing Structure 1 » 3 4 5 
education on drug use 
Process 1 2 3] 4 5 


The correct number is circled. 
1 = does not work 
5 = works very well 
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Annex 6: Consolidated tables 


This annex contains the results of the indicators exercise in the 12 countries 
which participated in the workshop. These results are presented following the 
list of indicators and in two sets: the first one from page 85 to 99 covers Bulgaria, 
Chad, Colombia, Guinea, India and Mali; the second one from page 100 to 
page 114 covers the Philippines, Sri Lanka, Thailand, Viet Nam, Zambia and 
Zimbabwe. : 


Most of these data were presented during the workshop. They were then 
scrutizined for relevance, consistency, and reliability by Mrs K. Timmermans 
and Mrs P. Brudon, who received answers to their queries from the principal 
investigators of each country. The new consolidated tables were sent again to the 


country teams which reviewed them and finally a double check took place in 
Geneva using the consolidated table and the report for each country. However, it 
should be noted that some of these results are preliminary (Chad and Zambia). 
Some are incomplete (Sri Lanka); some are difficult to present in a table as they 
are structured differently (Colombia). Finally, in a number of cases, data alone 
do not provide a sufficiently clear picture, and there is a need to revert to country 
reports which will be available at a later stage. 


The data were collected mainly at the end of 1995 and beginning of 1996. For 
data related to background information (BG1 to BG31), most of them refer to 
1994. 
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The WHO Action Programme on Essential Drugs seeks to ensure 
that all people, wherever they may be, are able to obtain the drugs 
they need at a price that they and their country can afford; that 
these drugs are safe, effective and of good quality; and that they 
are prescribed and used rationally. It provides operational support 
to countries in the development and implementation of national 
drug policies based on the concept of essential drugs and it 
promotes the rational use of drugs at every level. 
. 


Ensuring access to and rational use of drugs for all people is a 
difficult goal in itself. It is made even more complicated to achieve 
by rapidly changing macro-economic and national environments. 
Countries are experiencing the effects of international adjustment 
and stabilization policies; globalization of world markets; new 
disease patterns; widespread health system reforms with shifting 
priorities, and a changing relationship between the public and 
private sectors. Governments lack crucial information to guide 
their national drug policies in response to these challenges. 


Operational research makes a vital contribution to identifying 
global and national drug sector problems and priority areas for 
intervention. At global level, the systematic development and 
analysis of internationally comparable data on pharmaceutical 
systems strengthen national drug policy by enabling countries to 
learn from each other’s experience. At national level, research 
assists countries in analysing the constraints they face in developing 
and implementing drug policies and in gaining knowledge about the 
best means of selecting, procuring and distributing drugs, as well as 
the use of drugs by prescribers and consumers. The results of such 
operational research have a direct bearing on strategies to make 
vital medicines available and accessible to the greatest number of 
people. | 


This document is part of a series reporting on the activities and 
results of the Action Programme’s operational research. _ 
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